-- Logs begin at Wed 2022-07-27 07:45:52 UTC, end at Wed 2022-07-27 09:05:08 UTC. -- Jul 27 08:18:59.151589 dv-cinder-851170-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 27 08:18:59.161554 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 27 08:18:59 2022] *** Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: nodename: dv-cinder-851170-2-iscsi Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: machine: x86_64 Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: clock source: unix Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: pcre jit disabled Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: detected number of CPU cores: 4 Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: current working directory: / Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: detected binary path: /usr/bin/uwsgi-core Jul 27 08:18:59.164139 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: your processes number limit is 47680 Jul 27 08:18:59.164767 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: your memory page size is 4096 bytes Jul 27 08:18:59.164767 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: detected max file descriptor number: 2048 Jul 27 08:18:59.164767 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: lock engine: pthread robust mutexes Jul 27 08:18:59.164767 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: thunder lock: enabled Jul 27 08:18:59.164767 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 27 08:18:59.164767 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 27 08:18:59.203532 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: Python main interpreter initialized at 0x555e6825b9c0 Jul 27 08:18:59.203532 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: python threads support enabled Jul 27 08:18:59.203532 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: your server socket listen backlog is limited to 100 connections Jul 27 08:18:59.203532 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: your mercy for graceful operations on workers is 90 seconds Jul 27 08:18:59.203864 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: mapped 403077 bytes (393 KB) for 2 cores Jul 27 08:18:59.203954 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: *** Operational MODE: preforking *** Jul 27 08:18:59.204223 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: *** uWSGI is running in multiple interpreter mode *** Jul 27 08:18:59.204223 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: spawned uWSGI master process (pid: 80278) Jul 27 08:18:59.204424 dv-cinder-851170-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 27 08:18:59.209751 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: spawned uWSGI worker 1 (pid: 80290, cores: 1) Jul 27 08:18:59.209751 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: spawned uWSGI worker 2 (pid: 80291, cores: 1) Jul 27 08:18:59.209751 dv-cinder-851170-2-iscsi devstack@keystone.service[80278]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 27 08:19:04.744774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 08:19:04.744774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: from cryptography.utils import int_from_bytes Jul 27 08:19:04.745556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 08:19:04.745556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: from cryptography.utils import int_from_bytes Jul 27 08:19:04.745906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 08:19:04.745906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: from cryptography.utils import int_from_bytes Jul 27 08:19:04.746395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 27 08:19:04.746395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: from cryptography.utils import int_from_bytes Jul 27 08:19:04.795078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 08:19:04.795269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 08:19:04.795464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] command line args: [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 08:19:04.795735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 08:19:04.795867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 27 08:19:04.795956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ================================================================================ {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 08:19:04.796073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 27 08:19:04.796181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] admin_token = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.796259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] command line args: [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 27 08:19:04.796394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] config_dir = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.796539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 27 08:19:04.796619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.796721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ================================================================================ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 27 08:19:04.796810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] config_source = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.796917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] debug = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.796999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] admin_token = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.797114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.797431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config_dir = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.797431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.797631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.797732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config_source = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.797828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.797935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] debug = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] list_limit = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] log_config_append = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.798904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] log_dir = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] log_file = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_config_append = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_dir = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.799998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.800961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.801973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.802136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.802272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.802395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.802473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.802648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.802899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.803014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.803137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] publish_errors = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.803287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] publish_errors = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.804901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] standard_threads = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] standard_threads = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.805983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] use_journal = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] use_json = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] use_stderr = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] use_syslog = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_journal = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.806921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_json = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.807133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.807246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_stderr = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.807430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.807548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_syslog = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.807668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.807787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 27 08:19:04.807907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.808978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.809891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.810922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811511 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.811987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.812907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.813927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.814046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.814439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.814561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.814675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.814802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.814918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.815962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.816933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.817911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.818938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.819952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.820996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.821933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.822917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.823990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.824990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.825918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.826899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.827944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828536 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.828886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.829936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.830907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.831949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] auth.external = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.832902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] auth.password = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.external = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] auth.token = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.833996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.834111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.834242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.password = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.834357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.834992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.token = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.835137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.835296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.835415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.835531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.835646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.835772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.835888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] credential.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.836954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.837905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.838892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.839902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.840972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.841884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.842928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.843978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.844925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.845952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.846067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.846222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.846608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.846739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.846857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.846972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.847918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.848989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.849920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.850035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.850741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.850865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.850982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851799 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.851920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.852978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.853910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.854884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.855950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856535 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.856997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.857980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.858992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.859923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860622 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.860968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.861895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.862917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.863928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864515 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.864902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.865024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.865024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.865243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.865358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.865473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.865600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.865978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.866108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.866307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.866307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.866557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.866687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.866805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.866921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] role.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] role.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.867969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.868916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.869972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.870971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.871905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.872989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.873924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.874999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.875941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.876987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.877927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.878892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.connection = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.879957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.880933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.881051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.881175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.881373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.881517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.881517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.881780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.881897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.882936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.883991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80290) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 27 08:19:04.884233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 27 08:19:04.910517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.910750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.910979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.911295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.911527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.911752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.919980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.920889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.949681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.949911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.950573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:04.950812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.003879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.004164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.055352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.055584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.056378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.056620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.120968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.121197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.121424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.121510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.121668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.121751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.121894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.121978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.122111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.122295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.122417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.122576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.145215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.145449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.145653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.145808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.145908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.145987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.146106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.146211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.146334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.146416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.146550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.146635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.146796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.147586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.158056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.158557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.171343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.171578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.171798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.171912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80290) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.172832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.173051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.173274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 27 08:19:05.189457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.189667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 08:19:05.190006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.190247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 08:19:05.190354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.190471 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 08:19:05.190569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.190761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.190844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.190998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 27 08:19:05.191079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.191258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.191349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.191516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.191771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.192026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.201376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.201654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.201925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.202226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.202373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.202652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.202922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.203208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.209099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.210332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.213870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.214163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.214450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.214729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.215008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.215089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.215267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.215358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.215533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.215614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.215804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.215894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.216073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.216170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.216346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.216427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.216611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.216691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.216928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.217202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.217476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.217744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.240417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.241820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.242081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.242383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.243344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.243633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.246992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.247271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.247552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.248261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.248540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.248822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.254130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.254436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.255481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.255765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.259373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.259658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.259939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.260220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.260496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.260780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.260863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.261049 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.261137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.261324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.261405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.261627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.261905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.262208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.262489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.262765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.277449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.277728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.278749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.279030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.281643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.282974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.284824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.285117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.286158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.286466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.290058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.291455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.293236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.294822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.296505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.296804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.297091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.297382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.297670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.297957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.298038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.298352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.298645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.298944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.299228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.299517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.314537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.314819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.315086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.315349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.316285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.316571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.316839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.317101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.322384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.323928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.324209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.324304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.325795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.326071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.327030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.327308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.327605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.327887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.328174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.328911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.329192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.329489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.329780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.330071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.339623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.339903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.340183 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.340464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.340745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.341017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.341288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.341667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.341950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.342260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.342545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.342829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.343107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.343386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.357651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.359990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.360626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.363018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.363591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.366596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.366761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.368184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.368469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.368749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.369097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.370643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.370938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.371262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.375237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.376837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.377708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.379343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.379779 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.380070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.380353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.380638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.382434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.382724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.383018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.383292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.388817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.389105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.389396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.391442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.391733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.392013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.396238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80290) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.396516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.396796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.397109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.397390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.397672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.397945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.398246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.398619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.398901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.398986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 27 08:19:05.399182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.399266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.399456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.399548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.399731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.399814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.399990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80290) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.400073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.400285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.400562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.400829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.401200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.401484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.401766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.402045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.402338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.402602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 27 08:19:05.434461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555e6825b9c0 pid: 80290 (default app) Jul 27 08:19:05.436978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555e6825b9c0 pid: 80291 (default app) Jul 27 08:19:05.437763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a5c4d-76c8-4f7b-8e84-8ab5a27f1404 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:05.438055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a5c4d-76c8-4f7b-8e84-8ab5a27f1404 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:05.438355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a5c4d-76c8-4f7b-8e84-8ab5a27f1404 None None] PATH_INFO: `/v3` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:05.442614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Wed Jul 27 08:19:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:14.003914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a95fe-3224-4e22-8ea9-68911c757ab7 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:14.007835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a95fe-3224-4e22-8ea9-68911c757ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:14.007835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a95fe-3224-4e22-8ea9-68911c757ab7 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:14.009559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:14.014998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:14.015417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:14.015557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:14.086811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: INFO dbcounter [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] Registered counter for database keystone Jul 27 08:19:14.108198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG oslo_db.sqlalchemy.engines [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80290) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 08:19:14.114501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writer thread running {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 08:19:14.295454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG passlib.handlers.bcrypt [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80290) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 08:19:14.295801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG passlib.handlers.bcrypt [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80290) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 08:19:14.350332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:14.351963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:14.476572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 08:19:14.589997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a95fe-3224-4e22-8ea9-68911c757ab7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:14.659579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: INFO dbcounter [None req-8b5a95fe-3224-4e22-8ea9-68911c757ab7 None None] Registered counter for database keystone Jul 27 08:19:14.678947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG oslo_db.sqlalchemy.engines [None req-8b5a95fe-3224-4e22-8ea9-68911c757ab7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80291) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 27 08:19:14.685568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writer thread running {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 27 08:19:14.803815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a95fe-3224-4e22-8ea9-68911c757ab7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:14.808160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461eae9-f0ef-4b86-b54f-8ec13a716bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:14.808792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461eae9-f0ef-4b86-b54f-8ec13a716bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:14.809075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461eae9-f0ef-4b86-b54f-8ec13a716bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:14.809350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461eae9-f0ef-4b86-b54f-8ec13a716bcd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:14.817315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461eae9-f0ef-4b86-b54f-8ec13a716bcd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:14.870916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6461eae9-f0ef-4b86-b54f-8ec13a716bcd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 08:19:14.871262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461eae9-f0ef-4b86-b54f-8ec13a716bcd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:14.880710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Jul 27 08:19:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:19:14.886872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:14.901527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf409c4f-b407-4ff7-8585-1ff7b29f8c9f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:14.905822 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40833eae-6131-408c-9637-f2156a0fe53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:14.906944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40833eae-6131-408c-9637-f2156a0fe53e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:14.907228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40833eae-6131-408c-9637-f2156a0fe53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:14.907501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40833eae-6131-408c-9637-f2156a0fe53e admin admin] PATH_INFO: `/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:14.908958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40833eae-6131-408c-9637-f2156a0fe53e admin admin] RBAC: Authorizing `identity:get_project(project_id=28a37ee923b244c5bc101a0c6c5cd5d6)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:14.961148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING keystone.common.rbac_enforcer.enforcer [None req-40833eae-6131-408c-9637-f2156a0fe53e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 27 08:19:14.961486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40833eae-6131-408c-9637-f2156a0fe53e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:14.966007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Wed Jul 27 08:19:14 2022] GET /identity/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:16.967649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d9229-4389-4d63-ac56-ee51648d9839 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:16.968186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d9229-4389-4d63-ac56-ee51648d9839 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:16.968186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d9229-4389-4d63-ac56-ee51648d9839 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:16.972548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:16.977924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530129-324f-46d3-881c-e15cec835e12 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:16.978370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530129-324f-46d3-881c-e15cec835e12 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:16.978564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24530129-324f-46d3-881c-e15cec835e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:16.998771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-24530129-324f-46d3-881c-e15cec835e12 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:17.000277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-24530129-324f-46d3-881c-e15cec835e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:17.008676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 27 08:19:17.030686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d9229-4389-4d63-ac56-ee51648d9839 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:17.042782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d9229-4389-4d63-ac56-ee51648d9839 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:17.047079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f891812-eb1b-437e-b74c-b91b164585bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:17.047684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f891812-eb1b-437e-b74c-b91b164585bc admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:17.047965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f891812-eb1b-437e-b74c-b91b164585bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:17.048232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f891812-eb1b-437e-b74c-b91b164585bc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:17.056818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f891812-eb1b-437e-b74c-b91b164585bc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:17.058630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f891812-eb1b-437e-b74c-b91b164585bc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:17.067286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Jul 27 08:19:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 08:19:17.073239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24530129-324f-46d3-881c-e15cec835e12 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:17.082170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24530129-324f-46d3-881c-e15cec835e12 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:17.086638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c3874-13d7-4cc5-ab8b-19e0e0a3ee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:17.087263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c3874-13d7-4cc5-ab8b-19e0e0a3ee7b admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:17.087543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c3874-13d7-4cc5-ab8b-19e0e0a3ee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:17.087811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c3874-13d7-4cc5-ab8b-19e0e0a3ee7b admin admin] PATH_INFO: `/v3/users/c247fd0f0c364a67b9e7168fc38f6873` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:17.089148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c3874-13d7-4cc5-ab8b-19e0e0a3ee7b admin admin] RBAC: Authorizing `identity:get_user(user_id=c247fd0f0c364a67b9e7168fc38f6873)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:17.091006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c3874-13d7-4cc5-ab8b-19e0e0a3ee7b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:17.092521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Wed Jul 27 08:19:17 2022] GET /identity/v3/users/c247fd0f0c364a67b9e7168fc38f6873 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:19.142624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18037f-609c-407f-ba9b-83f85b6e4aab None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.143176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18037f-609c-407f-ba9b-83f85b6e4aab None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.143176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18037f-609c-407f-ba9b-83f85b6e4aab None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.144505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:19.149544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3a5fc-a8c4-426e-8128-c3183080af41 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.149934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3a5fc-a8c4-426e-8128-c3183080af41 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.150264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3a5fc-a8c4-426e-8128-c3183080af41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.165025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d019c03-eda6-4b7d-b729-fedc25a87e65 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.165310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d019c03-eda6-4b7d-b729-fedc25a87e65 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.165592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d019c03-eda6-4b7d-b729-fedc25a87e65 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.166941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:19.169547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-21c3a5fc-a8c4-426e-8128-c3183080af41 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:19.171197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-21c3a5fc-a8c4-426e-8128-c3183080af41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:19.172014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.172302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.172578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.214669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG passlib.handlers.bcrypt [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80291) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 27 08:19:19.215085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG passlib.handlers.bcrypt [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80291) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 27 08:19:19.250225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:19.262616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:19.264211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:19.273754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3a5fc-a8c4-426e-8128-c3183080af41 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:19.297283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:19.365941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:19.376497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3a5fc-a8c4-426e-8128-c3183080af41 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:19.386918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:19.409933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:19.410545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.410827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.411110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.423096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:19.431766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:19.432850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:19.433703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:19.438707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:19.489257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49981f92-bdd4-4132-bbb8-63dd9251a55b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:19.491680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:19.520725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:19.528812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:19.529684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.529971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.530289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.538798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b119a-e348-4c2e-afd2-e7ad3e3a3be4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:19.574884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:19.575502 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.575780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.576054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.576797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:19.577074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:19.578496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:19.590803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:19.596584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:19.598992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:19.600582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:19.699555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 08:19:19.699732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b23d7-1081-435c-8653-e88fe18b40a9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:19.704824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:19.707613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:19.734971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:19.735604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.735884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.736159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] PATH_INFO: `/v3/users/c247fd0f0c364a67b9e7168fc38f6873` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.737325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] RBAC: Authorizing `identity:get_user(user_id=c247fd0f0c364a67b9e7168fc38f6873)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:19.738871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:19.740639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/users/c247fd0f0c364a67b9e7168fc38f6873 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:19.745949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:19.760266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:19.804284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3739d4-df76-4e17-a943-e94722d985e7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:19.842533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:19.843407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.843683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.843950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.864661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:19.866905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:19:19.886718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:19.887784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:19.888780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:19:19.888780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:19:19.888780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:19:19.888780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:19.888780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 9/18] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:19.895103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:19.950717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:19.952271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29c3e3-1203-447e-b7a2-f711ed8c73c8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:19.985619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:19.986291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:19.986583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:19.986863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:19.988006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:19.989454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:19.991868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 10/19] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:19.994973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c08aa-4c92-4075-8d7d-08f81be91124 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:19.997488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:20.029041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018cc80-8a8f-413e-a2bf-2a1156b67c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:20.029650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018cc80-8a8f-413e-a2bf-2a1156b67c27 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:20.029932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018cc80-8a8f-413e-a2bf-2a1156b67c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:20.030234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018cc80-8a8f-413e-a2bf-2a1156b67c27 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:20.030921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018cc80-8a8f-413e-a2bf-2a1156b67c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:20.031222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018cc80-8a8f-413e-a2bf-2a1156b67c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:20.032595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018cc80-8a8f-413e-a2bf-2a1156b67c27 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:20.049436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 10/20] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:20.078744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:20.099199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266747-2866-4e81-9b50-0582027c2cb7 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:20.136596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f819ce-0421-4fe0-b900-bb0248d1a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:20.137230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f819ce-0421-4fe0-b900-bb0248d1a6be None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:20.137507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f819ce-0421-4fe0-b900-bb0248d1a6be None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:20.137777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f819ce-0421-4fe0-b900-bb0248d1a6be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:20.138688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f819ce-0421-4fe0-b900-bb0248d1a6be None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.domain.id=default, user.id=c247fd0f0c364a67b9e7168fc38f6873) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:20.138999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f819ce-0421-4fe0-b900-bb0248d1a6be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:20.140470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f819ce-0421-4fe0-b900-bb0248d1a6be None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:20.148380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 11/21] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Jul 27 08:19:19 2022] GET /identity/v3/role_assignments?user.id=c247fd0f0c364a67b9e7168fc38f6873&scope.domain.id=default&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:22.370548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75a8e7-d66a-4dc8-9d37-385f1c65dad4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:22.371070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75a8e7-d66a-4dc8-9d37-385f1c65dad4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:22.371070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75a8e7-d66a-4dc8-9d37-385f1c65dad4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:22.372413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 11/22] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:22.377600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab333de9-5263-4787-8577-db63d7e2751f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:22.377999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab333de9-5263-4787-8577-db63d7e2751f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:22.378196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab333de9-5263-4787-8577-db63d7e2751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:22.399835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ab333de9-5263-4787-8577-db63d7e2751f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:22.401042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ab333de9-5263-4787-8577-db63d7e2751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:22.508762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 12/23] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:22.541223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75a8e7-d66a-4dc8-9d37-385f1c65dad4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:22.693822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75a8e7-d66a-4dc8-9d37-385f1c65dad4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:22.716826 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:22.743416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73228dd4-7753-4298-b2fa-d91c6f1f0072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:22.744074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73228dd4-7753-4298-b2fa-d91c6f1f0072 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:22.744380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73228dd4-7753-4298-b2fa-d91c6f1f0072 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:22.744656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73228dd4-7753-4298-b2fa-d91c6f1f0072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:22.758213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73228dd4-7753-4298-b2fa-d91c6f1f0072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:22.759700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73228dd4-7753-4298-b2fa-d91c6f1f0072 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-73228dd4-7753-4298-b2fa-d91c6f1f0072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:19:22.779240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:22.780071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:22.780860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:19:22.780860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:19:22.780860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:19:22.780860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:22.780860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 12/24] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:22.788146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab333de9-5263-4787-8577-db63d7e2751f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:22.866929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8828aa-794d-4ac0-8140-23ddcd2a3deb None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:22.867559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8828aa-794d-4ac0-8140-23ddcd2a3deb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:22.868030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8828aa-794d-4ac0-8140-23ddcd2a3deb None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:22.869569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 13/25] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:22.887810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367fc52-c15c-408e-90d5-257b30e92e03 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:22.888476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367fc52-c15c-408e-90d5-257b30e92e03 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:22.888973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367fc52-c15c-408e-90d5-257b30e92e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:22.897232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:22.926221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b367fc52-c15c-408e-90d5-257b30e92e03 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:22.927643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b367fc52-c15c-408e-90d5-257b30e92e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:22.937885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab333de9-5263-4787-8577-db63d7e2751f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:22.993977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c14021-1ee8-405a-9648-80f97ad5caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:22.994549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c14021-1ee8-405a-9648-80f97ad5caba None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:22.994859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c14021-1ee8-405a-9648-80f97ad5caba None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:22.995217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c14021-1ee8-405a-9648-80f97ad5caba None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:22.995921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c14021-1ee8-405a-9648-80f97ad5caba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:22.996220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c14021-1ee8-405a-9648-80f97ad5caba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:22.997621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c14021-1ee8-405a-9648-80f97ad5caba None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:23.031557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 13/26] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:23.033682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef9ade-44d2-49dd-ba66-150b96e62ecf None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.034194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef9ade-44d2-49dd-ba66-150b96e62ecf None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.034676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef9ade-44d2-49dd-ba66-150b96e62ecf None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.041992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 14/27] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:23.043017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef9ade-44d2-49dd-ba66-150b96e62ecf None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:23.072418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 14/28] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:23.074267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a9e39-6f9d-4887-80c0-6e9f1bbafd58 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.075706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a9e39-6f9d-4887-80c0-6e9f1bbafd58 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.077000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a9e39-6f9d-4887-80c0-6e9f1bbafd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.114366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-009a9e39-6f9d-4887-80c0-6e9f1bbafd58 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:23.116107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-009a9e39-6f9d-4887-80c0-6e9f1bbafd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:23.279950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef9ade-44d2-49dd-ba66-150b96e62ecf None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:23.329687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 15/29] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:23.337068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a9e39-6f9d-4887-80c0-6e9f1bbafd58 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:23.340309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d059d15-4fb9-446a-b811-f58f9b52cdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:23.340983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d059d15-4fb9-446a-b811-f58f9b52cdad None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.341264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d059d15-4fb9-446a-b811-f58f9b52cdad None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.341537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d059d15-4fb9-446a-b811-f58f9b52cdad None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.342827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d059d15-4fb9-446a-b811-f58f9b52cdad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:23.343711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d059d15-4fb9-446a-b811-f58f9b52cdad None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:23.354191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:23.378640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:23.454711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a9e39-6f9d-4887-80c0-6e9f1bbafd58 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:23.491980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd1fc9-10fd-4694-8e9b-29c4bbaf28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:23.493009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1fc9-10fd-4694-8e9b-29c4bbaf28bb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.493490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1fc9-10fd-4694-8e9b-29c4bbaf28bb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.493986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd1fc9-10fd-4694-8e9b-29c4bbaf28bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.506874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd1fc9-10fd-4694-8e9b-29c4bbaf28bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-60dd1fc9-10fd-4694-8e9b-29c4bbaf28bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:23.514026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:23.514931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:23.515684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:23.517957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 16/30] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:23.519658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e4e45-f74e-4730-be03-7bbb17b8b967 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.520167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e4e45-f74e-4730-be03-7bbb17b8b967 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.520638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e4e45-f74e-4730-be03-7bbb17b8b967 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.523160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 17/31] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:23.528716 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e4e45-f74e-4730-be03-7bbb17b8b967 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:23.613041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 15/32] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:19:23 2022] POST /identity/v3/projects => generated 301 bytes in 570 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:23.615373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b850-9d8c-4bbd-89b6-1342fb79e880 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.616113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b850-9d8c-4bbd-89b6-1342fb79e880 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.616403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b850-9d8c-4bbd-89b6-1342fb79e880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.631347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:23.649640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-8401b850-9d8c-4bbd-89b6-1342fb79e880 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:23.651322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8401b850-9d8c-4bbd-89b6-1342fb79e880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:23.651949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e4e45-f74e-4730-be03-7bbb17b8b967 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:23.692486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda05b6-49b2-4dc2-8e1a-874b53bfa285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:23.694004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda05b6-49b2-4dc2-8e1a-874b53bfa285 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.694328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda05b6-49b2-4dc2-8e1a-874b53bfa285 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.694608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda05b6-49b2-4dc2-8e1a-874b53bfa285 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.695344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda05b6-49b2-4dc2-8e1a-874b53bfa285 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:23.696981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda05b6-49b2-4dc2-8e1a-874b53bfa285 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:23.736035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 16/33] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:23.737043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401b850-9d8c-4bbd-89b6-1342fb79e880 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:23.856352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401b850-9d8c-4bbd-89b6-1342fb79e880 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:23.872716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:23.889534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e807b8f-be0f-4822-809b-577c5d7f217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:23.890178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807b8f-be0f-4822-809b-577c5d7f217d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.890466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807b8f-be0f-4822-809b-577c5d7f217d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.890743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807b8f-be0f-4822-809b-577c5d7f217d None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.891464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e807b8f-be0f-4822-809b-577c5d7f217d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:23.891772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e807b8f-be0f-4822-809b-577c5d7f217d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:23.893286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e807b8f-be0f-4822-809b-577c5d7f217d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:23.902990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 17/34] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:23.904712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b686e-2145-4ead-9932-09d926485d8e None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.905008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b686e-2145-4ead-9932-09d926485d8e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.905287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b686e-2145-4ead-9932-09d926485d8e None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.906698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 18/35] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:23.910866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b686e-2145-4ead-9932-09d926485d8e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:23.918935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 18/36] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 27 08:19:23 2022] POST /identity/v3/roles => generated 219 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:23.920400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e950e53-ffa8-4ad4-a744-e98d025618fb None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:23.920691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e950e53-ffa8-4ad4-a744-e98d025618fb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:23.920972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e950e53-ffa8-4ad4-a744-e98d025618fb None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:23.922533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 19/37] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:23.923323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e950e53-ffa8-4ad4-a744-e98d025618fb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:24.034797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b686e-2145-4ead-9932-09d926485d8e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:24.046792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e950e53-ffa8-4ad4-a744-e98d025618fb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:24.074020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d8101-ddba-4d4d-8d10-0751a9fddb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:24.074672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d8101-ddba-4d4d-8d10-0751a9fddb66 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.074953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d8101-ddba-4d4d-8d10-0751a9fddb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.075230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d8101-ddba-4d4d-8d10-0751a9fddb66 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.075950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d8101-ddba-4d4d-8d10-0751a9fddb66 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:24.077400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d8101-ddba-4d4d-8d10-0751a9fddb66 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:24.088114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385f95e-6588-4b17-8ce8-0b1aed20b6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:24.088809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385f95e-6588-4b17-8ce8-0b1aed20b6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.089041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385f95e-6588-4b17-8ce8-0b1aed20b6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.089317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385f95e-6588-4b17-8ce8-0b1aed20b6c2 None admin] PATH_INFO: `/v3/users/c247fd0f0c364a67b9e7168fc38f6873` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.091113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385f95e-6588-4b17-8ce8-0b1aed20b6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=c247fd0f0c364a67b9e7168fc38f6873)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:24.093128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385f95e-6588-4b17-8ce8-0b1aed20b6c2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:24.094529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 20/38] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:23 2022] GET /identity/v3/users/c247fd0f0c364a67b9e7168fc38f6873 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:24.095652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c53a7-5679-4780-b605-2b30f3a14911 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.095933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c53a7-5679-4780-b605-2b30f3a14911 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.096448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c53a7-5679-4780-b605-2b30f3a14911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.121824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-368c53a7-5679-4780-b605-2b30f3a14911 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:24.121824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-368c53a7-5679-4780-b605-2b30f3a14911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:24.129283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:24.145909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 19/39] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 27 08:19:23 2022] POST /identity/v3/roles => generated 225 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:24.147212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73652-3ce2-49de-9f7f-9aa3c90d963c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.147501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73652-3ce2-49de-9f7f-9aa3c90d963c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.150638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac73652-3ce2-49de-9f7f-9aa3c90d963c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.172292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-dac73652-3ce2-49de-9f7f-9aa3c90d963c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:24.173536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dac73652-3ce2-49de-9f7f-9aa3c90d963c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:24.197482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 21/40] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:24.198655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c53a7-5679-4780-b605-2b30f3a14911 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:24.225614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:24.267307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 20/41] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:24.268635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac73652-3ce2-49de-9f7f-9aa3c90d963c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:24.328588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c53a7-5679-4780-b605-2b30f3a14911 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:24.366708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:24.367478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.367763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.368038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.369184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:24.370677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:24.372300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 22/42] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:24.373033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:24.380352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac73652-3ce2-49de-9f7f-9aa3c90d963c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:24.387934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:24.415172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:24.415789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.416068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.416343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.417054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:24.418495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:24.471564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3fd5a-6bf7-4863-b389-2a39f8a7d220 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:24.499619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 21/43] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 27 08:19:24 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:24.500579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:24.513360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:24.514399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.514691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.517126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.517126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:24.517930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:24.519687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 23/44] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:24.526567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:24.534370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:24.624848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:24.632483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821b1ae-8628-48d9-a4b7-3810c1b779ac None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:24.644647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9b7c6e-f609-45a3-9dcc-a669c783ec4a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:24.672413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88907cd2-e81b-402e-b354-017f62d6c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:24.673105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907cd2-e81b-402e-b354-017f62d6c4f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.673383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907cd2-e81b-402e-b354-017f62d6c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.673656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907cd2-e81b-402e-b354-017f62d6c4f8 None admin] PATH_INFO: `/v3/domains/default/users/c247fd0f0c364a67b9e7168fc38f6873/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.678919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88907cd2-e81b-402e-b354-017f62d6c4f8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c247fd0f0c364a67b9e7168fc38f6873, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:24.680217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88907cd2-e81b-402e-b354-017f62d6c4f8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:24.688976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90b08c-c86a-4f5a-a165-2fd34b4e3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:24.689605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b08c-c86a-4f5a-a165-2fd34b4e3654 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:24.689893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b08c-c86a-4f5a-a165-2fd34b4e3654 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:24.690195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90b08c-c86a-4f5a-a165-2fd34b4e3654 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:24.691700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90b08c-c86a-4f5a-a165-2fd34b4e3654 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:24.694943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90b08c-c86a-4f5a-a165-2fd34b4e3654 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:24.745220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 22/45] 10.222.0.40 () {58 vars in 1361 bytes} [Wed Jul 27 08:19:24 2022] PUT /identity/v3/domains/default/users/c247fd0f0c364a67b9e7168fc38f6873/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:24.793322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 24/46] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:19:24 2022] POST /identity/v3/projects => generated 312 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:19:26.735190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ae1b1-dde5-4b85-8632-d7ae61ddd3cb None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:26.735697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ae1b1-dde5-4b85-8632-d7ae61ddd3cb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:26.735778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ae1b1-dde5-4b85-8632-d7ae61ddd3cb None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:26.737098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 23/47] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:26.742949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a393f-30c2-4002-ac58-90f4f6552550 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:26.743389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a393f-30c2-4002-ac58-90f4f6552550 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:26.743513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a393f-30c2-4002-ac58-90f4f6552550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:26.762101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-122a393f-30c2-4002-ac58-90f4f6552550 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:26.763323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-122a393f-30c2-4002-ac58-90f4f6552550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:26.797038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffbe78-63c3-44e0-9abb-6987870d433a None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:26.797335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffbe78-63c3-44e0-9abb-6987870d433a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:26.797620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffbe78-63c3-44e0-9abb-6987870d433a None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:26.799030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 24/48] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:26.804024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271bd9d-1bf0-4ba7-9538-73d2bb4d571f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:26.804313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271bd9d-1bf0-4ba7-9538-73d2bb4d571f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:26.804595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271bd9d-1bf0-4ba7-9538-73d2bb4d571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:26.822050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a271bd9d-1bf0-4ba7-9538-73d2bb4d571f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:26.823203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a271bd9d-1bf0-4ba7-9538-73d2bb4d571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:26.828597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 25/49] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:26.851932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a393f-30c2-4002-ac58-90f4f6552550 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:26.890510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 25/50] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:26.912691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:26.913344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271bd9d-1bf0-4ba7-9538-73d2bb4d571f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:26.931322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:26.950190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a393f-30c2-4002-ac58-90f4f6552550 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:26.984007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:26.984718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:26.984954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:26.985223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:26.995173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:27.000228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:27.001062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:27.001805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 26/51] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:27.008262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:27.016247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271bd9d-1bf0-4ba7-9538-73d2bb4d571f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:27.052154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:27.052866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:27.053149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:27.053457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:27.054613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:27.056071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:27.057642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 26/52] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:27.063214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:27.107411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a053895-e3e3-4a30-ac7f-1db667c9ee6c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:27.131161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:27.141651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:27.142308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:27.142590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:27.142866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:27.143579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:27.143883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:27.145269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:27.150478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:27.157936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 27/53] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:27.163479 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:27.165275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3b3ef-7b43-457b-80ac-555d2ed32d9f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:27.198885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:27.199502 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:27.199787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:27.200066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:27.200752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:27.202403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:27.262572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc3e3d-a1d2-4854-b0f0-9b34604c3d91 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:27.295761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:27.296399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:27.296682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:27.296955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] PATH_INFO: `/v3/users/c247fd0f0c364a67b9e7168fc38f6873` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:27.298113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] RBAC: Authorizing `identity:get_user(user_id=c247fd0f0c364a67b9e7168fc38f6873)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:27.299596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:27.301179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 28/54] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:27 2022] GET /identity/v3/users/c247fd0f0c364a67b9e7168fc38f6873 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:27.307177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:27.318590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 27/55] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Jul 27 08:19:27 2022] POST /identity/v3/projects => generated 298 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:19:27.366442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:27.411893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a662d-8e15-4470-a683-2b804ab1224f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:27.450036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a00d0b-2ca0-4d70-b8a0-fa128415f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:27.450718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00d0b-2ca0-4d70-b8a0-fa128415f384 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:27.451003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00d0b-2ca0-4d70-b8a0-fa128415f384 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:27.451275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00d0b-2ca0-4d70-b8a0-fa128415f384 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:27.452465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00d0b-2ca0-4d70-b8a0-fa128415f384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:27.453989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00d0b-2ca0-4d70-b8a0-fa128415f384 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:27.455625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 29/56] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:27.461990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:27.564045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23d481-35aa-4904-a559-a5c163f561e2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:27.599336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6930f7-faa2-46db-9411-c77eeca0dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:27.599956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6930f7-faa2-46db-9411-c77eeca0dd51 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:27.600239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6930f7-faa2-46db-9411-c77eeca0dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:27.600515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6930f7-faa2-46db-9411-c77eeca0dd51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:27.601307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6930f7-faa2-46db-9411-c77eeca0dd51 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.domain.id=default, user.id=c247fd0f0c364a67b9e7168fc38f6873) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:27.601618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6930f7-faa2-46db-9411-c77eeca0dd51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:27.603108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6930f7-faa2-46db-9411-c77eeca0dd51 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:27.608818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 28/57] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Jul 27 08:19:27 2022] GET /identity/v3/role_assignments?user.id=c247fd0f0c364a67b9e7168fc38f6873&scope.domain.id=default&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:29.318720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb9549-2c1a-4973-b40b-030bb8ec1ec8 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:29.319241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb9549-2c1a-4973-b40b-030bb8ec1ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:29.319328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb9549-2c1a-4973-b40b-030bb8ec1ec8 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:29.320624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 30/58] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:29.325736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7634f-b2e6-4a26-b948-5fd36ed7e77a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:29.326132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7634f-b2e6-4a26-b948-5fd36ed7e77a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:29.326336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7634f-b2e6-4a26-b948-5fd36ed7e77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:29.332591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:29.344943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-24e7634f-b2e6-4a26-b948-5fd36ed7e77a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:29.346113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-24e7634f-b2e6-4a26-b948-5fd36ed7e77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:29.410489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 29/59] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:29.433565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb9549-2c1a-4973-b40b-030bb8ec1ec8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:29.532197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb9549-2c1a-4973-b40b-030bb8ec1ec8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:29.554720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:29.565420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bfc46-50a9-42ef-b663-3498e97207a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:29.566137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bfc46-50a9-42ef-b663-3498e97207a6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:29.566456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bfc46-50a9-42ef-b663-3498e97207a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:29.566738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bfc46-50a9-42ef-b663-3498e97207a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:29.568518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bfc46-50a9-42ef-b663-3498e97207a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:29.569980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bfc46-50a9-42ef-b663-3498e97207a6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:29.571688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 31/60] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:29.577110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7634f-b2e6-4a26-b948-5fd36ed7e77a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:29.672856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7634f-b2e6-4a26-b948-5fd36ed7e77a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:29.701840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:29.706502 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431235df-8841-47b5-96c2-10475dd3b8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:29.707119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431235df-8841-47b5-96c2-10475dd3b8a5 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:29.707405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431235df-8841-47b5-96c2-10475dd3b8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:29.707682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431235df-8841-47b5-96c2-10475dd3b8a5 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:29.708778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431235df-8841-47b5-96c2-10475dd3b8a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:29.710252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431235df-8841-47b5-96c2-10475dd3b8a5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:29.819643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 30/61] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:19:29 2022] POST /identity/v3/users => generated 275 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:32.939771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42ae4d-3ea3-4869-9bfe-9acae7365c8b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:32.940766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42ae4d-3ea3-4869-9bfe-9acae7365c8b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:32.941268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42ae4d-3ea3-4869-9bfe-9acae7365c8b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:32.943583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 32/62] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:32.949196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb46d2-4061-42ef-b382-1eb7dedca920 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:32.949606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb46d2-4061-42ef-b382-1eb7dedca920 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:32.949754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb46d2-4061-42ef-b382-1eb7dedca920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:32.975122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-45fb46d2-4061-42ef-b382-1eb7dedca920 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:32.976565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-45fb46d2-4061-42ef-b382-1eb7dedca920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:33.006425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4785-5875-468e-b12f-90141a6decd1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.007029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4785-5875-468e-b12f-90141a6decd1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.007500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4785-5875-468e-b12f-90141a6decd1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.009259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 33/63] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:33.014564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a454a-d147-426b-83f4-74acadece7c4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.015092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a454a-d147-426b-83f4-74acadece7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.015576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a454a-d147-426b-83f4-74acadece7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.059601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 31/64] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:33.060550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-909a454a-d147-426b-83f4-74acadece7c4 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:33.061768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-909a454a-d147-426b-83f4-74acadece7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:33.082791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb46d2-4061-42ef-b382-1eb7dedca920 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.188851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 34/65] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:33.228204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a454a-d147-426b-83f4-74acadece7c4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.249198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb46d2-4061-42ef-b382-1eb7dedca920 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:33.283009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:33.302553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1c8d9-3490-4134-a4ba-f041d239d50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:33.303448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1c8d9-3490-4134-a4ba-f041d239d50a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.303885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1c8d9-3490-4134-a4ba-f041d239d50a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.304309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1c8d9-3490-4134-a4ba-f041d239d50a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.315781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1c8d9-3490-4134-a4ba-f041d239d50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-70d1c8d9-3490-4134-a4ba-f041d239d50a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:33.322938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:33.324000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:33.324960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:33.327072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 32/66] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:33.328955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef4401-b0e7-4a07-9bcb-33c9e83eeb7f None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.329373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef4401-b0e7-4a07-9bcb-33c9e83eeb7f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.329794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef4401-b0e7-4a07-9bcb-33c9e83eeb7f None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.331194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 33/67] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:33.332660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee646f96-f5b5-4673-b284-9e69619213be None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.333765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee646f96-f5b5-4673-b284-9e69619213be None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.334534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee646f96-f5b5-4673-b284-9e69619213be None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.336122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 34/68] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:33.336122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee646f96-f5b5-4673-b284-9e69619213be None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.364833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a454a-d147-426b-83f4-74acadece7c4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:33.392264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:33.408795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f3b0b-4e42-42aa-b907-145a45528bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:33.409473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f3b0b-4e42-42aa-b907-145a45528bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.409925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f3b0b-4e42-42aa-b907-145a45528bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.410268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f3b0b-4e42-42aa-b907-145a45528bc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.422226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f3b0b-4e42-42aa-b907-145a45528bc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ab0f3b0b-4e42-42aa-b907-145a45528bc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:33.427999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:33.429050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:33.430006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:33.432997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 35/69] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:33.437379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318b49-a25e-4935-a8c0-9822dbabe05e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.437379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318b49-a25e-4935-a8c0-9822dbabe05e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.437379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed318b49-a25e-4935-a8c0-9822dbabe05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.452794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ed318b49-a25e-4935-a8c0-9822dbabe05e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:33.453337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee646f96-f5b5-4673-b284-9e69619213be None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:33.453988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ed318b49-a25e-4935-a8c0-9822dbabe05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:33.489648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362c5e0-68cd-4209-997f-51ebdb3f5842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:33.490215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362c5e0-68cd-4209-997f-51ebdb3f5842 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.490500 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362c5e0-68cd-4209-997f-51ebdb3f5842 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.490771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362c5e0-68cd-4209-997f-51ebdb3f5842 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.491495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362c5e0-68cd-4209-997f-51ebdb3f5842 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:33.491811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362c5e0-68cd-4209-997f-51ebdb3f5842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:33.493224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362c5e0-68cd-4209-997f-51ebdb3f5842 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:33.502037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 35/70] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:33.507768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a59b9-4e68-4c8c-bee9-3b6a37219a05 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.508063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a59b9-4e68-4c8c-bee9-3b6a37219a05 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.508337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a59b9-4e68-4c8c-bee9-3b6a37219a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.527000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-da1a59b9-4e68-4c8c-bee9-3b6a37219a05 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:33.528199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-da1a59b9-4e68-4c8c-bee9-3b6a37219a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:33.529929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 36/71] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:33.530886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318b49-a25e-4935-a8c0-9822dbabe05e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.577726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:33.618302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 36/72] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:33.620291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a59b9-4e68-4c8c-bee9-3b6a37219a05 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.659568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed318b49-a25e-4935-a8c0-9822dbabe05e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:33.670858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:33.697162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:33.697786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.698065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.698518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.699390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:33.699834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:33.701356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:33.715492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 37/73] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:33.716540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.728767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a59b9-4e68-4c8c-bee9-3b6a37219a05 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:33.763484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:33.764054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.764339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.764614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.777950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:33.779438 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:33.781067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 37/74] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:33.781825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.811249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:33.821072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696a8c2-e435-4e68-b53f-368223dd1c1f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:33.855075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e21556-225e-40dc-b280-bc06052064da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:33.855708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e21556-225e-40dc-b280-bc06052064da None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.855992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e21556-225e-40dc-b280-bc06052064da None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.856293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e21556-225e-40dc-b280-bc06052064da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.867447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e21556-225e-40dc-b280-bc06052064da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-96e21556-225e-40dc-b280-bc06052064da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:33.872355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:33.873266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:33.874058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 38/75] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:33.874774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e21556-225e-40dc-b280-bc06052064da None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.886945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f298f-1b6a-4691-a627-77fdae4472af None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:33.907351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:33.922112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee3d01-f794-4888-a0ce-3b775e5ef6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:33.922799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee3d01-f794-4888-a0ce-3b775e5ef6ec None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.923121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee3d01-f794-4888-a0ce-3b775e5ef6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.923454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee3d01-f794-4888-a0ce-3b775e5ef6ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.934209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee3d01-f794-4888-a0ce-3b775e5ef6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-a6ee3d01-f794-4888-a0ce-3b775e5ef6ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:33.939094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:33.940002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:33.940769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 38/76] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:33.941447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f7638-0071-414e-8866-4d9fa410365b None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:33.941743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f7638-0071-414e-8866-4d9fa410365b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:33.942014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f7638-0071-414e-8866-4d9fa410365b None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:33.944045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 39/77] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:33.944762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f7638-0071-414e-8866-4d9fa410365b None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:33.978200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e21556-225e-40dc-b280-bc06052064da None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.011502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.012136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.012419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.012696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] PATH_INFO: `/v3/users/c247fd0f0c364a67b9e7168fc38f6873` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.013927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] RBAC: Authorizing `identity:get_user(user_id=c247fd0f0c364a67b9e7168fc38f6873)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.015351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.021970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 39/78] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/users/c247fd0f0c364a67b9e7168fc38f6873 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.021970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.035658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.049395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f7638-0071-414e-8866-4d9fa410365b None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.083703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc1df7-cb11-42a3-a494-761c17ec4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.084410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc1df7-cb11-42a3-a494-761c17ec4af1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.084695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc1df7-cb11-42a3-a494-761c17ec4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.084971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc1df7-cb11-42a3-a494-761c17ec4af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.115526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.118727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c685d93-eaaf-472d-aa17-31f274794452 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.152899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc1df7-cb11-42a3-a494-761c17ec4af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.154480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecc1df7-cb11-42a3-a494-761c17ec4af1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.161966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.162630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.162918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.163196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.163913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:34.164221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.165584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.173352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 40/79] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.174174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.243185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.255922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 40/80] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:34.257255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857901c-5ac3-4990-b53b-f4941506aa7e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.257553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857901c-5ac3-4990-b53b-f4941506aa7e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.257836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857901c-5ac3-4990-b53b-f4941506aa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.273150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05051bc-84e3-44d2-b84b-16a5f7eb1176 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.275607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-8857901c-5ac3-4990-b53b-f4941506aa7e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:34.276813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8857901c-5ac3-4990-b53b-f4941506aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:34.306551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.307170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.307458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.307732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.308439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:34.308746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.310185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.318729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 41/81] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.319545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.344943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 41/82] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:34.345783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857901c-5ac3-4990-b53b-f4941506aa7e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.349186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.429127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c1474-bd0a-4e63-a6a9-c7d661eb7b8f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.451674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857901c-5ac3-4990-b53b-f4941506aa7e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.461559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.462198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.462481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.462765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.468232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.485064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.485700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.485982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.486295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.487615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.489080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.491566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 42/83] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.492146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.532863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.534307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.561213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.592584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa3c45-e434-4406-b3c4-bed3cc0bdb86 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.625701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.626369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.626648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.626920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.628059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.629506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.631271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 43/84] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.632113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.635792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 42/85] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:34.636533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.689891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.734398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c0a5-3d28-44f8-a617-20770b238615 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.737121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba23bd-bfe0-4afe-86e4-4a12d8f816c9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.760985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.768017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.768659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.768944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.773057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.773057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.773308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.773402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.773482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.773482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.773482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.773660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 44/86] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.773894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.774240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.775421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.777074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 43/87] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.777911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.878039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604507b-403c-4e14-995c-c32e2414c5fb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.878446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc21c9-e968-45e6-bf09-5b947b624cfc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:34.900597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:34.912672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.912834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:34.913282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:34.913561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.913670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:34.913833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.913947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:34.914655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=1545cd354e894182b36f434125235edd, user.id=ac3564120871420cae8d959903f7be82) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:34.914964 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.916392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:34.924568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 44/88] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/role_assignments?user.id=ac3564120871420cae8d959903f7be82&scope.project.id=1545cd354e894182b36f434125235edd&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:34.924568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:34.995163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:34.996805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.000687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:35.061349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137826b0-923e-469f-87dd-bebfad3766c4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.102447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.103094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.103380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.103658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.104824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.106306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.107989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 45/89] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:35.108721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:35.117993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 45/90] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:35.118674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:35.146241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:35.215143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae84b8-8084-4acb-b42f-1e0f26c3bbbe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.230353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd25bac-a7a1-4fb5-a0b5-44745f1fdb25 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.240424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:35.251580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.252220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.252477 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.252751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.267295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.267922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.268203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.268480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.269167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.270703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.321496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.323006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.345339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 46/91] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:19:35 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:35.346459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:35.446564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:35.451453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 46/92] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:35.452559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:35.476889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea5173-6977-43ba-af30-f75cef3939f1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.480301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:35.515802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.516722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.516887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.517169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.517974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=1545cd354e894182b36f434125235edd, user.id=c247fd0f0c364a67b9e7168fc38f6873) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:35.518318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.520051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.527198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 47/93] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:35 2022] GET /identity/v3/role_assignments?user.id=c247fd0f0c364a67b9e7168fc38f6873&scope.project.id=1545cd354e894182b36f434125235edd&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:35.536683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:35.580191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff9a3d-8b29-421e-b120-81f13a5f3742 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.627458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.628504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.629047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.629522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.631256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.633095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.635240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 47/94] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:35 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:35.644741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:35.673349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249908d-d2a0-4bd4-927e-4600c0004cf3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.713829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.714836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.715324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.715801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] PATH_INFO: `/v3/projects/b2f02e083e4d4caf879c691101514dd5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.717227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] RBAC: Authorizing `identity:get_project(project_id=b2f02e083e4d4caf879c691101514dd5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.718920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.720843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 48/95] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:35 2022] GET /identity/v3/projects/b2f02e083e4d4caf879c691101514dd5 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:35.727152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:35.750525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:35.769944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:35.782310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c730d7-f6dd-49a5-939b-288a89175faa None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.821351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fbbb9-8069-4149-9e33-5cbd8d9b280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.821985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fbbb9-8069-4149-9e33-5cbd8d9b280f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.822330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fbbb9-8069-4149-9e33-5cbd8d9b280f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.822608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fbbb9-8069-4149-9e33-5cbd8d9b280f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.823507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fbbb9-8069-4149-9e33-5cbd8d9b280f None admin] RBAC: Adding query filter params (role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=1545cd354e894182b36f434125235edd, user.id=ac3564120871420cae8d959903f7be82) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:35.824135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fbbb9-8069-4149-9e33-5cbd8d9b280f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.825300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fbbb9-8069-4149-9e33-5cbd8d9b280f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.832184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 48/96] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:35 2022] GET /identity/v3/role_assignments?user.id=ac3564120871420cae8d959903f7be82&scope.project.id=1545cd354e894182b36f434125235edd&role.id=d469c6f0fa524f36949e61ae51807884 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:35.854849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41573b8c-9899-457f-86c5-0b73314a8536 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:35.892963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbe6d3-498b-4527-a2d7-1aac3c5a3fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:35.894067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe6d3-498b-4527-a2d7-1aac3c5a3fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:35.894603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe6d3-498b-4527-a2d7-1aac3c5a3fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:35.895082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbe6d3-498b-4527-a2d7-1aac3c5a3fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:35.896645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbe6d3-498b-4527-a2d7-1aac3c5a3fa5 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=b2f02e083e4d4caf879c691101514dd5, user.id=ac3564120871420cae8d959903f7be82) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:35.897181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbe6d3-498b-4527-a2d7-1aac3c5a3fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:35.898935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbe6d3-498b-4527-a2d7-1aac3c5a3fa5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:35.905548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 49/97] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:35 2022] GET /identity/v3/role_assignments?user.id=ac3564120871420cae8d959903f7be82&scope.project.id=b2f02e083e4d4caf879c691101514dd5&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:37.356804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082e6c-72cf-48bd-84f2-5865c396fd09 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:37.357310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082e6c-72cf-48bd-84f2-5865c396fd09 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:37.357381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b082e6c-72cf-48bd-84f2-5865c396fd09 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:37.358761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 49/98] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:37.367508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683b442-a932-4897-85f6-f3e2a2302899 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:37.367963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683b442-a932-4897-85f6-f3e2a2302899 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:37.368142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683b442-a932-4897-85f6-f3e2a2302899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:37.403185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6683b442-a932-4897-85f6-f3e2a2302899 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:37.404402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6683b442-a932-4897-85f6-f3e2a2302899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:37.500422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 50/99] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:37.554643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082e6c-72cf-48bd-84f2-5865c396fd09 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:37.679826 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b082e6c-72cf-48bd-84f2-5865c396fd09 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:37.725565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:37.726591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:37.727067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:37.727531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:37.738098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:37.747232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:37.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:37.748830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 50/100] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:37.755789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683b442-a932-4897-85f6-f3e2a2302899 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:37.771006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:37.887272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683b442-a932-4897-85f6-f3e2a2302899 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:37.922414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a04d4-e7e7-4bc6-9cc0-183adf1283b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:37.923042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a04d4-e7e7-4bc6-9cc0-183adf1283b3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:37.923326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a04d4-e7e7-4bc6-9cc0-183adf1283b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:37.923601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a04d4-e7e7-4bc6-9cc0-183adf1283b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:37.924332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a04d4-e7e7-4bc6-9cc0-183adf1283b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:37.924637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a04d4-e7e7-4bc6-9cc0-183adf1283b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:37.926038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a04d4-e7e7-4bc6-9cc0-183adf1283b3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:37.935735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 51/101] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:37.941263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:37.966985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:38.161493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7eeae-cd4f-4b58-9c80-a0d5a3cfc021 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:38.174240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b1297-49f2-475a-aa7a-d41e54c7440c None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.174795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b1297-49f2-475a-aa7a-d41e54c7440c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.175301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b1297-49f2-475a-aa7a-d41e54c7440c None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.177860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 52/102] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:38.195811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5dc28-7feb-428f-b14e-2a573e2d0e59 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.196332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5dc28-7feb-428f-b14e-2a573e2d0e59 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.196805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5dc28-7feb-428f-b14e-2a573e2d0e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.220743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-75c5dc28-7feb-428f-b14e-2a573e2d0e59 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:38.222666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-75c5dc28-7feb-428f-b14e-2a573e2d0e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:38.236695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:38.237606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.238066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.238562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.242421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:38.244083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:38.246003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 51/103] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:37 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:38.251830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:38.344305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:38.354723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 53/104] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 27 08:19:38.391128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5dc28-7feb-428f-b14e-2a573e2d0e59 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:38.406763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:38.430612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f3d08-0200-4cf2-8f37-7fccfd3c260b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:38.512078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:38.512660 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.512941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.513219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.514496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:38.516062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:38.517741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 52/105] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:38 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:38.524552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:38.624046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5dc28-7feb-428f-b14e-2a573e2d0e59 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:38.706877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:38.707868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.708363 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.708814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.722366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf927c2-4926-4973-9bfc-717d98f0370e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:38.730879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:38.736145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:38.736984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:38.737756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:38.746170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 54/106] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:38.747288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:38.810204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fc1eb-25f3-4d2b-b830-9fa2915ce065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:38.810918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fc1eb-25f3-4d2b-b830-9fa2915ce065 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.811205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fc1eb-25f3-4d2b-b830-9fa2915ce065 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.811480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fc1eb-25f3-4d2b-b830-9fa2915ce065 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd/users/ac3564120871420cae8d959903f7be82/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.821039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:38.821654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fc1eb-25f3-4d2b-b830-9fa2915ce065 None admin] RBAC: Authorizing `identity:create_grant(project_id=1545cd354e894182b36f434125235edd, user_id=ac3564120871420cae8d959903f7be82, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:38.823260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fc1eb-25f3-4d2b-b830-9fa2915ce065 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:38.868121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4fbc7-94a2-4c81-8caf-076dffdaf218 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:38.903981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017841a6-e97e-4dfa-9624-453349046258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:38.904656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017841a6-e97e-4dfa-9624-453349046258 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.904975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017841a6-e97e-4dfa-9624-453349046258 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.905249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017841a6-e97e-4dfa-9624-453349046258 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.905971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017841a6-e97e-4dfa-9624-453349046258 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:38.906344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017841a6-e97e-4dfa-9624-453349046258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:38.907741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017841a6-e97e-4dfa-9624-453349046258 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:38.917283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 55/107] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:38.918609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64e26b-9635-4e91-99df-573353d9b8c7 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.918918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64e26b-9635-4e91-99df-573353d9b8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.919177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64e26b-9635-4e91-99df-573353d9b8c7 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.924599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 56/108] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:38.925403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64e26b-9635-4e91-99df-573353d9b8c7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:38.940570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 53/109] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:38 2022] PUT /identity/v3/projects/1545cd354e894182b36f434125235edd/users/ac3564120871420cae8d959903f7be82/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:19:38.941744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb5d22-c400-4209-a95c-c65dd6fe9208 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:38.942777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb5d22-c400-4209-a95c-c65dd6fe9208 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:38.943062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb5d22-c400-4209-a95c-c65dd6fe9208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:38.983371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d0cb5d22-c400-4209-a95c-c65dd6fe9208 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:38.984709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d0cb5d22-c400-4209-a95c-c65dd6fe9208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:38.996764 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.064040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64e26b-9635-4e91-99df-573353d9b8c7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.087569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.105582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 54/110] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:39.109752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534060fe-ce37-4ad0-a9a4-74b7ed91666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.110547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534060fe-ce37-4ad0-a9a4-74b7ed91666d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.110829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534060fe-ce37-4ad0-a9a4-74b7ed91666d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.111104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534060fe-ce37-4ad0-a9a4-74b7ed91666d None admin] PATH_INFO: `/v3/users/c247fd0f0c364a67b9e7168fc38f6873` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.112581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534060fe-ce37-4ad0-a9a4-74b7ed91666d None admin] RBAC: Authorizing `identity:get_user(user_id=c247fd0f0c364a67b9e7168fc38f6873)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:39.112931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942adeb-8360-4986-ad26-de6f93c3537c None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.113233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942adeb-8360-4986-ad26-de6f93c3537c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.113518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942adeb-8360-4986-ad26-de6f93c3537c None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.114757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534060fe-ce37-4ad0-a9a4-74b7ed91666d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:39.116930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 57/111] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:38 2022] GET /identity/v3/users/c247fd0f0c364a67b9e7168fc38f6873 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:39.117992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 55/112] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:39 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:39.118211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dabf0-6e11-4fcb-b24f-a0b41a278860 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.118456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dabf0-6e11-4fcb-b24f-a0b41a278860 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.118729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dabf0-6e11-4fcb-b24f-a0b41a278860 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.120221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 58/113] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:39.128056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dabf0-6e11-4fcb-b24f-a0b41a278860 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.128619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d6012-df11-492d-9da0-a399421b7cd1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.129112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d6012-df11-492d-9da0-a399421b7cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.130197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d6012-df11-492d-9da0-a399421b7cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.153450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d98d6012-df11-492d-9da0-a399421b7cd1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:39.154747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d98d6012-df11-492d-9da0-a399421b7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:39.234323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 56/114] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:39.234323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9981f-72e4-4f6a-b6ce-5f7f54bdb599 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.234323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9981f-72e4-4f6a-b6ce-5f7f54bdb599 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.234323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9981f-72e4-4f6a-b6ce-5f7f54bdb599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.252417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dabf0-6e11-4fcb-b24f-a0b41a278860 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.258531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-06a9981f-72e4-4f6a-b6ce-5f7f54bdb599 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:39.259720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-06a9981f-72e4-4f6a-b6ce-5f7f54bdb599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:39.290520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.291164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.291452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.291729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.293228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:39.295156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:39.297521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 59/115] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:39 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:39.298219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.344537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 57/116] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:39.344537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9ccf7-ee68-4fdc-90e3-6f1a05f5035a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.344537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9ccf7-ee68-4fdc-90e3-6f1a05f5035a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.344537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9ccf7-ee68-4fdc-90e3-6f1a05f5035a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.344537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 58/117] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:39.344537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9ccf7-ee68-4fdc-90e3-6f1a05f5035a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.357922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.370373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.419534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a28073-bca6-4feb-829f-2db4348e9a2f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.454814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.455679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.456108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.456380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.460380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9ccf7-ee68-4fdc-90e3-6f1a05f5035a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.468461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:39.474640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:39.475596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 60/118] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:39.476695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.503526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6706ad-a8eb-4be6-95d5-8e2953b7caf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.504177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6706ad-a8eb-4be6-95d5-8e2953b7caf2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.504546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6706ad-a8eb-4be6-95d5-8e2953b7caf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.505495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6706ad-a8eb-4be6-95d5-8e2953b7caf2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.506910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6706ad-a8eb-4be6-95d5-8e2953b7caf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:39.508469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6706ad-a8eb-4be6-95d5-8e2953b7caf2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:39.510240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 59/119] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:39.511359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31765c7-b490-49f1-8d7b-e4e24a2bf2d6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.511651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31765c7-b490-49f1-8d7b-e4e24a2bf2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.511930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31765c7-b490-49f1-8d7b-e4e24a2bf2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.530549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b31765c7-b490-49f1-8d7b-e4e24a2bf2d6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:39.531779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b31765c7-b490-49f1-8d7b-e4e24a2bf2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:39.592177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e9cf0-48b8-42c9-b5dd-ce47a9c35311 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.604534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.605907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.609919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 60/120] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:39.611208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31765c7-b490-49f1-8d7b-e4e24a2bf2d6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.633639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.634961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.635258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.635539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd/users/c247fd0f0c364a67b9e7168fc38f6873/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.637546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1545cd354e894182b36f434125235edd, user_id=c247fd0f0c364a67b9e7168fc38f6873, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:39.639067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:39.712811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31765c7-b490-49f1-8d7b-e4e24a2bf2d6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.721906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 61/121] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:39 2022] PUT /identity/v3/projects/1545cd354e894182b36f434125235edd/users/c247fd0f0c364a67b9e7168fc38f6873/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:39.722775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.773787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.775533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.776088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.777043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.794085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:39.802360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:39.803271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:39.804027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:39.807321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 61/122] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:19:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:39.808852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.849806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2288-ff46-4fc4-b3b7-d655c44904e2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.895101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.895667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.895951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.896512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.896617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.897281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:39.897616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:39.898999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:39.908517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 62/123] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:39.909344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:39.941164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393cc82-27c2-4c18-ac87-fecd7d2b2211 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:39.955196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:39.982796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:39.983422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:39.983705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:39.984017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:39.984664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:39.986133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:40.032711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed070226-16c3-46f1-a9c9-f58174901dc4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:40.066285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:40.066994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:40.067775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:40.068050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:40.078481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:40.084831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:40.090917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 63/124] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:40.091672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.154199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 62/125] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:19:39 2022] POST /identity/v3/projects => generated 302 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:40.154990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.265068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:40.276514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1931f6-fc72-47e3-b2f6-27854165bf7d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:40.338800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c70c2-db10-4137-b21a-b7018472c5c1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:40.339255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:40.339888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:40.340174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:40.340451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:40.341177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:40.341486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:40.343013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:40.353666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:40.356456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 64/126] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:19:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:40.357154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.386603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:40.387492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:40.388365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:40.388821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:40.392575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:40.394689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:40.398414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 63/127] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:40 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:40.399770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.488523 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd279b-3f2a-49ac-8dc7-ed24c1bb1567 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:40.533707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf8f2b-9e27-439f-bdda-f8583d69203c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:40.534727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:40.535412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:40.536305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:40.536676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:40.537655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] RBAC: Adding query filter params (name=reader) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:40.537970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:40.540306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:40.550455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 65/128] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:40.551914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.579889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:40.580333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:40.581633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:40.581927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:40.583980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:40.585571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:40.587921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:40.590211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 64/129] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:40 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:40.591288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.682533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:40.730488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58143a9-5ca8-4f1a-b579-0949f49239be None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:40.765172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada143de-3123-41b0-b5c1-3d2e162c559c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:40.804504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:40.805302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:40.805589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:40.805870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] PATH_INFO: `/v3/users/c103f5ff760246efba69fb05c2bdc851` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:40.816468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:40.817693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:40.818507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:40.818806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] PATH_INFO: `/v3/projects/b2f02e083e4d4caf879c691101514dd5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:40.821283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] RBAC: Authorizing `identity:get_project(project_id=b2f02e083e4d4caf879c691101514dd5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:40.822783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:40.824491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 66/130] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:40 2022] GET /identity/v3/projects/b2f02e083e4d4caf879c691101514dd5 => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:40.828551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.830688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=c103f5ff760246efba69fb05c2bdc851)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:40.832172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:40.834343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 65/131] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:40 2022] GET /identity/v3/users/c103f5ff760246efba69fb05c2bdc851 => generated 289 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:40.836891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:40.967831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:40.995459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ada345-92a9-4aea-ac59-230c38f457b4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:41.042523 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:41.043104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.043392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.043677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.044863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:41.046353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:41.048144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 67/132] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:40 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:41.053291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:41.068604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db91d54-a65f-4147-b294-bb0ff0494eb7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:41.080601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:41.155767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:41.156820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.157302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.157777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] PATH_INFO: `/v3/projects/b2f02e083e4d4caf879c691101514dd5/users/ac3564120871420cae8d959903f7be82/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.163467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f02e083e4d4caf879c691101514dd5, user_id=ac3564120871420cae8d959903f7be82, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:41.165140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:41.167714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853f99-caaf-4850-b50e-d0e9989a497a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:41.205612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48511c-bc08-467a-b708-b06bef5c58aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:41.206370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48511c-bc08-467a-b708-b06bef5c58aa None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.206664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48511c-bc08-467a-b708-b06bef5c58aa None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.206999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48511c-bc08-467a-b708-b06bef5c58aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.235800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:41.236945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 66/133] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:40 2022] PUT /identity/v3/projects/b2f02e083e4d4caf879c691101514dd5/users/ac3564120871420cae8d959903f7be82/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:41.237964 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:41.287250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48511c-bc08-467a-b708-b06bef5c58aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:41.289103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48511c-bc08-467a-b708-b06bef5c58aa None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:41.445888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 68/134] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:41.451347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de540c-6693-434c-b041-26a383bda284 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.451644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de540c-6693-434c-b041-26a383bda284 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.453294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de540c-6693-434c-b041-26a383bda284 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.453294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 69/135] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:41.456039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de540c-6693-434c-b041-26a383bda284 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:41.471413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05146982-7218-4c19-b4fe-7ba1fc6b633a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:41.509404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd718b-280e-45fb-9249-31f62b171459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:41.510092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd718b-280e-45fb-9249-31f62b171459 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.510406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd718b-280e-45fb-9249-31f62b171459 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.510687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd718b-280e-45fb-9249-31f62b171459 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd/users/ac3564120871420cae8d959903f7be82/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.514170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd718b-280e-45fb-9249-31f62b171459 None admin] RBAC: Authorizing `identity:create_grant(project_id=1545cd354e894182b36f434125235edd, user_id=ac3564120871420cae8d959903f7be82, role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:41.515636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd718b-280e-45fb-9249-31f62b171459 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:41.535271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:41.582361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de540c-6693-434c-b041-26a383bda284 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:41.592610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 67/136] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:41 2022] PUT /identity/v3/projects/1545cd354e894182b36f434125235edd/users/ac3564120871420cae8d959903f7be82/roles/d469c6f0fa524f36949e61ae51807884 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:41.593987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e3c1e-a28f-4ecb-b23a-7cac981336dc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.594316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e3c1e-a28f-4ecb-b23a-7cac981336dc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.594648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e3c1e-a28f-4ecb-b23a-7cac981336dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.623014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:41.623803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.624094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.624369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.625675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:41.627274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:41.629196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 70/137] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:41 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:41.639345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:41.642956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f36e3c1e-a28f-4ecb-b23a-7cac981336dc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:41.644394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f36e3c1e-a28f-4ecb-b23a-7cac981336dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:41.683280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:41.773487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 68/138] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:41.815108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46da3d5-6192-4948-9871-ff58a36807e6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:41.864221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:41.866092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e3c1e-a28f-4ecb-b23a-7cac981336dc None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:41.867135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:41.867760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:41.868042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:41.868314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:41.869113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] RBAC: Adding query filter params (role.id=ffc01dcb00834115860b2bf1f7390419, scope.project.id=1545cd354e894182b36f434125235edd, user.id=c103f5ff760246efba69fb05c2bdc851) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:41.869431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:41.871044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:41.876866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 71/139] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:41 2022] GET /identity/v3/role_assignments?user.id=c103f5ff760246efba69fb05c2bdc851&scope.project.id=1545cd354e894182b36f434125235edd&role.id=ffc01dcb00834115860b2bf1f7390419 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:41.998683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e3c1e-a28f-4ecb-b23a-7cac981336dc None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:42.046889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2862c-006b-4352-a3a3-492a5f7608c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:42.046889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2862c-006b-4352-a3a3-492a5f7608c1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.047322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2862c-006b-4352-a3a3-492a5f7608c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.047488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2862c-006b-4352-a3a3-492a5f7608c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.059759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2862c-006b-4352-a3a3-492a5f7608c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4df2862c-006b-4352-a3a3-492a5f7608c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:42.064792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:42.067844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:42.069751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 69/140] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:42.078683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:42.125047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608f8c-9e7b-4ec7-bc1a-8cb7acebc947 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.125352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608f8c-9e7b-4ec7-bc1a-8cb7acebc947 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.125613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c608f8c-9e7b-4ec7-bc1a-8cb7acebc947 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.128018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 70/141] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:42.133984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5cade-4198-4938-8da8-246498ee25ff None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.134316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5cade-4198-4938-8da8-246498ee25ff None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.134603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5cade-4198-4938-8da8-246498ee25ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.171191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-56c5cade-4198-4938-8da8-246498ee25ff None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:42.172759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-56c5cade-4198-4938-8da8-246498ee25ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:42.202856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:42.237369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f85f88-02b0-40f5-bbd2-4348d1ac55bd None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:42.278040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:42.278685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.278964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.279233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.279950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:42.280256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:42.281621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:42.294419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 72/142] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:42.296626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:42.357005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:42.372433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 71/143] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:42.395939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5cade-4198-4938-8da8-246498ee25ff None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:42.416335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ddffd-6b8d-415b-8b11-6f969ed051c3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:42.487170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:42.488155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.488430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.488777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.490403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:42.493339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:42.497355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 73/144] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:42 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:42.508250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:42.618734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5cade-4198-4938-8da8-246498ee25ff None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:42.671006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f8747-31c0-49b6-a9b3-17d0ded79bd4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:42.688153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a3b16-f4a1-40eb-99bc-3a0885d48609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:42.689113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a3b16-f4a1-40eb-99bc-3a0885d48609 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.689612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a3b16-f4a1-40eb-99bc-3a0885d48609 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.690088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a3b16-f4a1-40eb-99bc-3a0885d48609 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.705192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:42.712466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a3b16-f4a1-40eb-99bc-3a0885d48609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4a3a3b16-f4a1-40eb-99bc-3a0885d48609 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:42.717630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:42.722810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 72/145] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:42.724288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cc1dd-0384-4925-9e51-3b31d36081df None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.724963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cc1dd-0384-4925-9e51-3b31d36081df None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.724963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cc1dd-0384-4925-9e51-3b31d36081df None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.731289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 73/146] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:42.732026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cc1dd-0384-4925-9e51-3b31d36081df None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:42.739608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:42.741205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:42.746915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:42.747432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:42.748916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:42.891134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:42.892552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:42.929356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cc1dd-0384-4925-9e51-3b31d36081df None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:43.007784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37291864-9f60-40a6-9bfd-a6ea5dafdc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:43.008898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37291864-9f60-40a6-9bfd-a6ea5dafdc51 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:43.009397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37291864-9f60-40a6-9bfd-a6ea5dafdc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:43.009875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37291864-9f60-40a6-9bfd-a6ea5dafdc51 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:43.011653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37291864-9f60-40a6-9bfd-a6ea5dafdc51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:43.014664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37291864-9f60-40a6-9bfd-a6ea5dafdc51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:43.016306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37291864-9f60-40a6-9bfd-a6ea5dafdc51 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:43.040132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 74/147] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:43.041392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c4ce4-77db-451c-ae25-4fdc39fcc71d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:43.041685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c4ce4-77db-451c-ae25-4fdc39fcc71d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:43.041960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c4ce4-77db-451c-ae25-4fdc39fcc71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:43.058561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 74/148] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:43.060552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:43.079708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-684c4ce4-77db-451c-ae25-4fdc39fcc71d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:43.080917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-684c4ce4-77db-451c-ae25-4fdc39fcc71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:43.091914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:43.167670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0828f97-ad8f-4f5e-b529-be8e3518bf6f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:43.211627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:43.212633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:43.213112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:43.213583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] PATH_INFO: `/v3/users/c247fd0f0c364a67b9e7168fc38f6873` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:43.214972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] RBAC: Authorizing `identity:get_user(user_id=c247fd0f0c364a67b9e7168fc38f6873)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:43.218003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:43.220570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 75/149] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:43 2022] GET /identity/v3/users/c247fd0f0c364a67b9e7168fc38f6873 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:43.223930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:43.253659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:43.264227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 75/150] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:43.268692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c4ce4-77db-451c-ae25-4fdc39fcc71d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:43.413853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:43.445023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0105445-b997-49c0-b3f6-b434dbb8a592 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:43.452417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c4ce4-77db-451c-ae25-4fdc39fcc71d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:43.499958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:43.500907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:43.501384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:43.503003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:43.520761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:43.521692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:43.522174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:43.522648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:43.525488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:43.527304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:43.532953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 76/151] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:43 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:43.534043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:43.649438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:43.659158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:43.665908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:43.685950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b465c1-4fa5-40b2-8f31-d0f206df555f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:43.733914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:43.734896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:43.735356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:43.735800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:43.738321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:43.740802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:43.743578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 77/152] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:43.745215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:43.807921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:43.860746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 76/153] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:43.863122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:43.976618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba80d3-487b-48f7-811d-c64155331a55 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:43.990586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:43.995736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3f0b1-669f-4bf2-8195-261a59979fb9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:44.040472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:44.041431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.041881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.043349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.044396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=1545cd354e894182b36f434125235edd, user.id=ac3564120871420cae8d959903f7be82) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:44.044902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:44.048129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:44.064588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 78/154] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:43 2022] GET /identity/v3/role_assignments?user.id=ac3564120871420cae8d959903f7be82&scope.project.id=1545cd354e894182b36f434125235edd&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:44.070778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:44.091017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83d191-cdc1-4efa-a17b-b8c53c5b079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:44.091959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83d191-cdc1-4efa-a17b-b8c53c5b079a None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.092411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83d191-cdc1-4efa-a17b-b8c53c5b079a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.093581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83d191-cdc1-4efa-a17b-b8c53c5b079a None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.094549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83d191-cdc1-4efa-a17b-b8c53c5b079a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:44.096630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83d191-cdc1-4efa-a17b-b8c53c5b079a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:44.234410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 77/155] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:19:43 2022] POST /identity/v3/users => generated 283 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:44.235724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f9694-93df-40fb-a0bd-3bd859418c89 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.236052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f9694-93df-40fb-a0bd-3bd859418c89 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.236294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f9694-93df-40fb-a0bd-3bd859418c89 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.237954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 78/156] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:44.264572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b670-ddd8-4650-9267-7cf7363a24e0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.264572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b670-ddd8-4650-9267-7cf7363a24e0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.264572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b670-ddd8-4650-9267-7cf7363a24e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.290583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:44.291833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6114b670-ddd8-4650-9267-7cf7363a24e0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:44.293061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6114b670-ddd8-4650-9267-7cf7363a24e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:44.318159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7115f-d362-4101-99b7-86197eddd785 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:44.361182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:44.361812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.362090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.363444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.363712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:44.366370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:44.370068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 79/157] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:44 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:44.376365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:44.419890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 79/158] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:44.431738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62487bea-3490-42b5-8143-4fe1a6159c7c None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.432062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62487bea-3490-42b5-8143-4fe1a6159c7c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.432298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62487bea-3490-42b5-8143-4fe1a6159c7c None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.433962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 80/159] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:44.453377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f220-a1a4-4a55-b84b-65033f766025 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.453661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f220-a1a4-4a55-b84b-65033f766025 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.453936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f220-a1a4-4a55-b84b-65033f766025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.474712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ee19f220-a1a4-4a55-b84b-65033f766025 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:44.475926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ee19f220-a1a4-4a55-b84b-65033f766025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:44.522877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146b98f-edb8-4b64-befc-9e44acb182db None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:44.562996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90426bb-541d-4949-9f02-d2c3daabbc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:44.563618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90426bb-541d-4949-9f02-d2c3daabbc12 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.563897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90426bb-541d-4949-9f02-d2c3daabbc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.564165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90426bb-541d-4949-9f02-d2c3daabbc12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.564971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90426bb-541d-4949-9f02-d2c3daabbc12 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=1545cd354e894182b36f434125235edd, user.id=c247fd0f0c364a67b9e7168fc38f6873) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:44.565279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90426bb-541d-4949-9f02-d2c3daabbc12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:44.567394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90426bb-541d-4949-9f02-d2c3daabbc12 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:44.568701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 81/160] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:44.570010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f220-a1a4-4a55-b84b-65033f766025 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:44.587583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:44.588799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 80/161] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:44 2022] GET /identity/v3/role_assignments?user.id=c247fd0f0c364a67b9e7168fc38f6873&scope.project.id=1545cd354e894182b36f434125235edd&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:44.590195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859d56e-c5f9-408a-846b-6c9436e35472 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.590480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859d56e-c5f9-408a-846b-6c9436e35472 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.590749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859d56e-c5f9-408a-846b-6c9436e35472 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.592223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 81/162] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:44.599827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3f5f2-c18e-47b9-881c-27d743b0eb76 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.602261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3f5f2-c18e-47b9-881c-27d743b0eb76 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.602261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3f5f2-c18e-47b9-881c-27d743b0eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.622317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9cd3f5f2-c18e-47b9-881c-27d743b0eb76 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:44.622729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9cd3f5f2-c18e-47b9-881c-27d743b0eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:44.651417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:44.726628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f220-a1a4-4a55-b84b-65033f766025 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:44.729555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 82/163] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:44.730900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3f5f2-c18e-47b9-881c-27d743b0eb76 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:44.772754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:44.773472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.773695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.773968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.789659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:44.796250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:44.797194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:44.797979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 82/164] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:44.799309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:44.871737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3f5f2-c18e-47b9-881c-27d743b0eb76 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:44.897878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:44.913965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:44.914711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.914989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.915259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.927623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16e242-b37a-4579-a4b8-edc8f7bf2591 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:44.930552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:44.935536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:44.936422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:44.937190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 83/165] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:19:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:44.938066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:44.951347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:44.976351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:44.977003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:44.977284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:44.977716 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:44.990766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:44.996662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:45.003393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 83/166] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:45.004846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:45.081928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a081f-9b2f-4e1e-bdb1-c13c572cdb48 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:45.121619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714e7cf-2e26-4ce7-8a27-9a17d234a3fa None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:45.126354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:45.126980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:45.127262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:45.127514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:45.128249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:45.128553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:45.129997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:45.141550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 84/167] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:45.142908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:45.174989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:45.175623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:45.175901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:45.176745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:45.177482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:45.177782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:45.179212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:45.182063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:45.196576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 84/168] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:19:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:45.198083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:45.247257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:45.275240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d756c6f-d0bf-4662-9755-8e2dcf04114d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:45.319671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:45.320307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:45.320591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:45.322415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:45.323154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:45.323461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:45.324858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:45.345378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 85/169] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:45.349841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:45.437318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60d5b4-89d1-468c-80d2-4f240740f987 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:45.475223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31224cc-ba8a-451c-bac8-ac7caa0708e1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:45.477036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:45.477911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:45.478440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:45.478919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:45.480317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:45.482019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:45.483908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 85/170] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:45 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:45.485077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:45.526504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:45.537603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:45.538355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:45.538565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:45.538883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] PATH_INFO: `/v3/users/ac3564120871420cae8d959903f7be82` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:45.541972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] RBAC: Authorizing `identity:get_user(user_id=ac3564120871420cae8d959903f7be82)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:45.543481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:45.545463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 86/171] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:45 2022] GET /identity/v3/users/ac3564120871420cae8d959903f7be82 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:45.546072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:45.575511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:45.644187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb68a34-9291-45b1-85a0-3150a97df4a0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:45.697012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd0102-9f82-4e56-a0ea-9f9572aad550 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:45.732549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:45.733857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:45.734331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:45.734845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:45.750862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:45.752078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:45.752534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:45.752958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] PATH_INFO: `/v3/users/c103f5ff760246efba69fb05c2bdc851` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:45.759011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] RBAC: Authorizing `identity:get_user(user_id=c103f5ff760246efba69fb05c2bdc851)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:45.760657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:45.762475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 86/172] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:45 2022] GET /identity/v3/users/c103f5ff760246efba69fb05c2bdc851 => generated 289 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:45.763379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:45.806521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:45.807867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:45.809530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:45.915985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 87/173] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:45.917164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:46.005704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaea795-2202-4c54-aa33-6100a7e24beb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:46.018983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23021481-3a5d-4fff-afa0-0a6203e49fe7 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:46.046513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:46.059274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:46.060138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:46.060673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:46.062360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:46.063050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:46.065074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:46.067684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:46.071725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 88/174] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:45 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:46.072640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:46.075674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:46.075674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:46.075674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:46.075922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:46.210017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:46.211477 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:46.249258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac21933-2bf7-4631-9505-2d2e7932fa27 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:46.284641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:46.286350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:46.286834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:46.287321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] PATH_INFO: `/v3/projects/b2f02e083e4d4caf879c691101514dd5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:46.288684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] RBAC: Authorizing `identity:get_project(project_id=b2f02e083e4d4caf879c691101514dd5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:46.290364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:46.292288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 89/175] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:46 2022] GET /identity/v3/projects/b2f02e083e4d4caf879c691101514dd5 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:46.293248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:46.329349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:46.384437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 87/176] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:46.386984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:46.447745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:46.451841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a1e4d-388a-4253-afd8-2414a00a8728 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:46.522780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:46.523778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:46.524257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:46.524742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd/users/c103f5ff760246efba69fb05c2bdc851/roles/ffc01dcb00834115860b2bf1f7390419` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:46.526945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] RBAC: Authorizing `identity:create_grant(project_id=1545cd354e894182b36f434125235edd, user_id=c103f5ff760246efba69fb05c2bdc851, role_id=ffc01dcb00834115860b2bf1f7390419)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:46.528863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:46.568117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435cf7a-70e4-4e43-995e-0ffc43c972ba None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:46.604044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 90/177] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:46 2022] PUT /identity/v3/projects/1545cd354e894182b36f434125235edd/users/c103f5ff760246efba69fb05c2bdc851/roles/ffc01dcb00834115860b2bf1f7390419 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:46.605001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:46.626486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:46.627113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:46.627391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:46.627659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:46.628458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=b2f02e083e4d4caf879c691101514dd5, user.id=ac3564120871420cae8d959903f7be82) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:46.628767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:46.632252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:46.646912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 88/178] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:46 2022] GET /identity/v3/role_assignments?user.id=ac3564120871420cae8d959903f7be82&scope.project.id=b2f02e083e4d4caf879c691101514dd5&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:46.743777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:46.751850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae314d-d139-4dc9-8c5c-167a2ef25fcc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:46.797666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914b348-640e-4764-b021-392aa87a998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:46.798305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914b348-640e-4764-b021-392aa87a998d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:46.798561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914b348-640e-4764-b021-392aa87a998d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:46.798847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914b348-640e-4764-b021-392aa87a998d None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:46.800027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914b348-640e-4764-b021-392aa87a998d None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:46.802138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914b348-640e-4764-b021-392aa87a998d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:46.804075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 91/179] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:46 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:46.810606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:46.928899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:46.935902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e62ba-2c12-4a92-a49f-ee744204aeb1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:46.976884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa789ddb-c87e-47e4-a602-3c8489791e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:46.977941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789ddb-c87e-47e4-a602-3c8489791e05 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:46.978381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789ddb-c87e-47e4-a602-3c8489791e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:46.978661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789ddb-c87e-47e4-a602-3c8489791e05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:46.979465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa789ddb-c87e-47e4-a602-3c8489791e05 None admin] RBAC: Adding query filter params (role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=1545cd354e894182b36f434125235edd, user.id=ac3564120871420cae8d959903f7be82) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:46.979781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa789ddb-c87e-47e4-a602-3c8489791e05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:46.981224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa789ddb-c87e-47e4-a602-3c8489791e05 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:46.988845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 89/180] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:46 2022] GET /identity/v3/role_assignments?user.id=ac3564120871420cae8d959903f7be82&scope.project.id=1545cd354e894182b36f434125235edd&role.id=d469c6f0fa524f36949e61ae51807884 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:47.159352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb079ac8-e1cd-404a-ac6b-6ecb1b366626 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.159720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb079ac8-e1cd-404a-ac6b-6ecb1b366626 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.159990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb079ac8-e1cd-404a-ac6b-6ecb1b366626 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.161896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 92/181] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:47.169773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23546602-5980-4fbf-8758-b728d6243ce8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.170020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23546602-5980-4fbf-8758-b728d6243ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.170502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23546602-5980-4fbf-8758-b728d6243ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.192381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-23546602-5980-4fbf-8758-b728d6243ce8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:47.193807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-23546602-5980-4fbf-8758-b728d6243ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:47.275831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 90/182] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:47.299223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb079ac8-e1cd-404a-ac6b-6ecb1b366626 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:47.409186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5ef65-d9cc-4662-a60e-09dc19f85bcc None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.409547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5ef65-d9cc-4662-a60e-09dc19f85bcc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.409955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5ef65-d9cc-4662-a60e-09dc19f85bcc None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.411647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 91/183] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:47.412700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3872124-004a-4a68-8779-26ad3ff3289a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.412999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3872124-004a-4a68-8779-26ad3ff3289a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.413255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3872124-004a-4a68-8779-26ad3ff3289a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.418065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 92/184] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:47.419131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb3b1-3b83-495a-af27-9f2fd61f5b2b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.419482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb3b1-3b83-495a-af27-9f2fd61f5b2b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.419765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8deb3b1-3b83-495a-af27-9f2fd61f5b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.434942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb079ac8-e1cd-404a-ac6b-6ecb1b366626 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:47.438685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c8deb3b1-3b83-495a-af27-9f2fd61f5b2b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:47.440867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c8deb3b1-3b83-495a-af27-9f2fd61f5b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:47.476164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba426487-bff6-4496-bb8e-6fbcf8d35bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:47.476800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba426487-bff6-4496-bb8e-6fbcf8d35bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.477078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba426487-bff6-4496-bb8e-6fbcf8d35bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.477546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba426487-bff6-4496-bb8e-6fbcf8d35bd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.489125 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba426487-bff6-4496-bb8e-6fbcf8d35bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-ba426487-bff6-4496-bb8e-6fbcf8d35bd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:47.495288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:47.496175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:47.496996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 93/185] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:47.498541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8182fa5-2dc5-44bf-806e-ba878ddb0944 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.498541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8182fa5-2dc5-44bf-806e-ba878ddb0944 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.498541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8182fa5-2dc5-44bf-806e-ba878ddb0944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.517317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b8182fa5-2dc5-44bf-806e-ba878ddb0944 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:47.518538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b8182fa5-2dc5-44bf-806e-ba878ddb0944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:47.525542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 93/186] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:47.527254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8deb3b1-3b83-495a-af27-9f2fd61f5b2b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:47.539165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:47.553762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:47.599767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 94/187] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:47.600626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8182fa5-2dc5-44bf-806e-ba878ddb0944 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:47.647773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8deb3b1-3b83-495a-af27-9f2fd61f5b2b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:47.683596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:47.684148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.684426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.684700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.685411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:47.685718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:47.687124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:47.701380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 94/188] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:47.701380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:47.708670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8182fa5-2dc5-44bf-806e-ba878ddb0944 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:47.745274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:47.746071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.746440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.746716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.758021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:47.762992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:47.763863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:47.764698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 95/189] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:19:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:47.765344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:47.784116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:47.799020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:47.819992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aadfa0-b646-4d1e-a281-301c51133ada None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:47.856714 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:47.857467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.857661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.857935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.859116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:47.860560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:47.862203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 95/190] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:47.867920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:47.879866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669ac40-dd3e-4eda-9ac4-6619bfebd077 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:47.918444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:47.919074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:47.919377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:47.919626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] PATH_INFO: `/v3/users/74f1d05277d147dcb9f686a36e3efbf5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:47.935009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] RBAC: Authorizing `identity:get_user(user_id=74f1d05277d147dcb9f686a36e3efbf5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:47.936428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:47.938238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 96/191] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:47 2022] GET /identity/v3/users/74f1d05277d147dcb9f686a36e3efbf5 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:47.944096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:47.976670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f705a7b-2176-4fe4-85ca-492ceb420dc4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:48.013660 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:48.014305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:48.014609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:48.014875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:48.015574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:48.015883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:48.017257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:48.018102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:48.026870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 96/192] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:19:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:48.027579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:48.052012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0435c48-c017-40b6-92d5-a64547707818 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:48.127574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:48.148690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:48.149555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:48.149968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:48.150410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:48.151239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:48.152822 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:48.218318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf50e78-d6c4-455a-82af-4abfea546410 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:48.251998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 97/193] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:19:47 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:48.253086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:48.259010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:48.260064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:48.260614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:48.261102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:48.357024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:48.358563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:48.374298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:48.386398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4085f7-e2c2-4fc4-af0a-44195765c021 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:48.421772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:48.433077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:48.433723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:48.434006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:48.434352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] PATH_INFO: `/v3/users/74f1d05277d147dcb9f686a36e3efbf5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:48.435593 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] RBAC: Authorizing `identity:get_user(user_id=74f1d05277d147dcb9f686a36e3efbf5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:48.437057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:48.438938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 98/194] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:48 2022] GET /identity/v3/users/74f1d05277d147dcb9f686a36e3efbf5 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:48.445189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:48.486764 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 97/195] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:48.492345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:48.588357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0bcb0-3eb6-406c-9061-b02983b56bea None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:48.630453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:48.631347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:48.631637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:48.631914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:48.633459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d2033-6503-48dd-9bfc-1bb85f2c9fba None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:48.651284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:48.674318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:48.674973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:48.675251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:48.675524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:48.677581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:48.679068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:48.681053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 98/196] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:48 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:48.686954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:48.710364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:48.714965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:48.716483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:48.822555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3b9d5-1624-493b-b0ee-c2a1351e45e8 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:48.838687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 99/197] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:48.847192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:48.895865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:48.897345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:48.897942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:48.899545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:48.901199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=74f1d05277d147dcb9f686a36e3efbf5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:48.902266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:48.903986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:48.913524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 99/198] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:48 2022] GET /identity/v3/role_assignments?user.id=74f1d05277d147dcb9f686a36e3efbf5&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:48.996737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:49.003826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea298533-1f5f-49f9-8fb7-1b37a3528c8d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:49.056906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1b258-1bc2-43fe-97e0-78769a9768d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:49.057834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b258-1bc2-43fe-97e0-78769a9768d2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:49.058341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b258-1bc2-43fe-97e0-78769a9768d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:49.058750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b258-1bc2-43fe-97e0-78769a9768d2 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:49.062964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1b258-1bc2-43fe-97e0-78769a9768d2 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:49.064908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1b258-1bc2-43fe-97e0-78769a9768d2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:49.067442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 100/199] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:48 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:49.073288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:49.135712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:49.170384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e62746-755f-46e4-8bef-55a88e386582 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:49.171000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e62746-755f-46e4-8bef-55a88e386582 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:49.171468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e62746-755f-46e4-8bef-55a88e386582 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:49.173011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 101/200] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:19:49.185368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1097d2-dec0-424d-adc9-f12ea1f858cf None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:49.186388 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1097d2-dec0-424d-adc9-f12ea1f858cf None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:49.187020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1097d2-dec0-424d-adc9-f12ea1f858cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:49.207492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e431efa-e34d-4354-b266-9d2735cb961f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:49.220622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fa1097d2-dec0-424d-adc9-f12ea1f858cf None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:49.222453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fa1097d2-dec0-424d-adc9-f12ea1f858cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:49.261326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:49.262015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:49.262320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:49.262597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:49.263402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] RBAC: Adding query filter params (role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=74f1d05277d147dcb9f686a36e3efbf5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:49.263715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:49.265177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:49.278319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 100/201] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:49 2022] GET /identity/v3/role_assignments?user.id=74f1d05277d147dcb9f686a36e3efbf5&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=d469c6f0fa524f36949e61ae51807884 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:49.344964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 102/202] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:49.389161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:49.507700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dcb53-d0f8-4b39-8018-c91973c4431a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:49.516981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:49.542654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:49.543300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:49.543607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:49.543886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:49.555163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:49.560197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:49.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:49.561844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 101/203] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:49.571052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1097d2-dec0-424d-adc9-f12ea1f858cf None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:49.723436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:49.729933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1097d2-dec0-424d-adc9-f12ea1f858cf None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:49.785796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:49.786453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:49.786733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:49.787008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:49.787799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:49.788108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:49.789499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:49.803954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 103/204] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:49.809536 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:49.916373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56d2f5-a9f2-4e88-bea3-4220a3098678 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:49.952112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:49.952767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:49.953050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:49.953375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] PATH_INFO: `/v3/users/c103f5ff760246efba69fb05c2bdc851` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:49.954590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] RBAC: Authorizing `identity:get_user(user_id=c103f5ff760246efba69fb05c2bdc851)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:49.956047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:49.958044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 102/205] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:49 2022] GET /identity/v3/users/c103f5ff760246efba69fb05c2bdc851 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:49.963493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:50.108931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8098d4-1967-427f-8141-a56bd3fd7d64 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:50.164305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:50.165751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:50.166032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:50.166330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:50.201553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:50.257460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:50.258942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:50.367720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 104/206] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:50.373282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:50.412909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:50.482236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea422f1b-4ef8-4dc2-a071-2b0f163e3873 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:50.520851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935211d-e1f8-4bba-ba78-5252e2ada561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:50.521531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935211d-e1f8-4bba-ba78-5252e2ada561 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:50.521816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935211d-e1f8-4bba-ba78-5252e2ada561 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:50.522093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935211d-e1f8-4bba-ba78-5252e2ada561 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:50.523297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935211d-e1f8-4bba-ba78-5252e2ada561 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:50.524758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935211d-e1f8-4bba-ba78-5252e2ada561 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:50.526756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 103/207] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:50 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:50.535799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:50.630187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:50.676361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472f68-2e8a-402c-bd24-c876bc3842e4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:50.717698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dad19-f95f-4d73-bc18-71080e3d87e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:50.718727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dad19-f95f-4d73-bc18-71080e3d87e1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:50.719222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dad19-f95f-4d73-bc18-71080e3d87e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:50.719708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dad19-f95f-4d73-bc18-71080e3d87e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:50.720715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dad19-f95f-4d73-bc18-71080e3d87e1 None admin] RBAC: Adding query filter params (role.id=ffc01dcb00834115860b2bf1f7390419, scope.project.id=1545cd354e894182b36f434125235edd, user.id=c103f5ff760246efba69fb05c2bdc851) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:50.721482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dad19-f95f-4d73-bc18-71080e3d87e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:50.723213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dad19-f95f-4d73-bc18-71080e3d87e1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:50.734865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 105/208] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:50 2022] GET /identity/v3/role_assignments?user.id=c103f5ff760246efba69fb05c2bdc851&scope.project.id=1545cd354e894182b36f434125235edd&role.id=ffc01dcb00834115860b2bf1f7390419 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:50.978874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a749a-690f-49f7-8af8-43bcd5513aef None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:50.979352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a749a-690f-49f7-8af8-43bcd5513aef None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:50.979352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a749a-690f-49f7-8af8-43bcd5513aef None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:50.981065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 104/209] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:50.986315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf70b-f7e2-42c5-8d93-78a3627a9339 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:50.986722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf70b-f7e2-42c5-8d93-78a3627a9339 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:50.986905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf70b-f7e2-42c5-8d93-78a3627a9339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.007295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b39cf70b-f7e2-42c5-8d93-78a3627a9339 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:51.010462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b39cf70b-f7e2-42c5-8d93-78a3627a9339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:51.086284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 106/210] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:51.107721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a749a-690f-49f7-8af8-43bcd5513aef None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:51.144055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360bfd6-93b1-4f85-8e1e-20c27d606753 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.144419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360bfd6-93b1-4f85-8e1e-20c27d606753 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.144632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360bfd6-93b1-4f85-8e1e-20c27d606753 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.146178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 107/211] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:51.151357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5af53b-6782-4480-92f1-b1a6d613145e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.151641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5af53b-6782-4480-92f1-b1a6d613145e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.151914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5af53b-6782-4480-92f1-b1a6d613145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.171270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-dc5af53b-6782-4480-92f1-b1a6d613145e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:51.172453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dc5af53b-6782-4480-92f1-b1a6d613145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:51.234744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a749a-690f-49f7-8af8-43bcd5513aef None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:51.236179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:51.238682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:51.269484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 108/212] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:51.276651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:51.277721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.277721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.277881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.278996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:51.280512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:51.282206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 105/213] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:51.291305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5af53b-6782-4480-92f1-b1a6d613145e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:51.293579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:51.411652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5af53b-6782-4480-92f1-b1a6d613145e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:51.424472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee17754-aa86-46c4-a90c-f51c6e810bf6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:51.449879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96af67-8529-4a05-8342-f3392b2a5d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:51.450920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96af67-8529-4a05-8342-f3392b2a5d7c None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.451410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96af67-8529-4a05-8342-f3392b2a5d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.451881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96af67-8529-4a05-8342-f3392b2a5d7c None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.452770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96af67-8529-4a05-8342-f3392b2a5d7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:51.454636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96af67-8529-4a05-8342-f3392b2a5d7c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:51.465118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:51.466175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.466484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.466757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.478773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:51.484189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:51.485081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:51.485885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:51.487233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 106/214] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:51.493134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:51.526945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:51.545844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 109/215] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:19:51 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:51.547376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f8d6-a61b-47ee-82fe-0d4400ce41a1 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.547660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f8d6-a61b-47ee-82fe-0d4400ce41a1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.554622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66f8d6-a61b-47ee-82fe-0d4400ce41a1 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.558188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 110/216] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:51.561741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba495e5f-2dee-4597-b8b2-983a8714a5c2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.562102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba495e5f-2dee-4597-b8b2-983a8714a5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.562320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba495e5f-2dee-4597-b8b2-983a8714a5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.607276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ba495e5f-2dee-4597-b8b2-983a8714a5c2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:51.608479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ba495e5f-2dee-4597-b8b2-983a8714a5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:51.612526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da616cef-ec99-4e09-a07f-543a6079ce24 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:51.650478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:51.652883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:51.653425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.653714 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.653992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.654798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:51.655108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:51.656545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:51.667049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 107/217] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:51.672611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:51.714171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 111/218] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:51.732693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba495e5f-2dee-4597-b8b2-983a8714a5c2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:51.792176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:51.799920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88414235-4c08-4d72-ac51-f3004e3fe9a1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:51.849793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84835c-4695-4a2f-be30-92b6c6794529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:51.850475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84835c-4695-4a2f-be30-92b6c6794529 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.850761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84835c-4695-4a2f-be30-92b6c6794529 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.851038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84835c-4695-4a2f-be30-92b6c6794529 None admin] PATH_INFO: `/v3/users/f97a9cac4cc6489fb979b5c1a95f6139` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.856496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba495e5f-2dee-4597-b8b2-983a8714a5c2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:51.872857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84835c-4695-4a2f-be30-92b6c6794529 None admin] RBAC: Authorizing `identity:get_user(user_id=f97a9cac4cc6489fb979b5c1a95f6139)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:51.874390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84835c-4695-4a2f-be30-92b6c6794529 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:51.876154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 108/219] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:51 2022] GET /identity/v3/users/f97a9cac4cc6489fb979b5c1a95f6139 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:51.878288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb225d-ca98-4ead-89ee-1b251fd516c6 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.878574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb225d-ca98-4ead-89ee-1b251fd516c6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.879454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb225d-ca98-4ead-89ee-1b251fd516c6 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.881065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 109/220] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:51.886445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c49f16-1552-4a76-a8ab-5db4be497d6c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.886724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c49f16-1552-4a76-a8ab-5db4be497d6c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.886998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c49f16-1552-4a76-a8ab-5db4be497d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.908871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-00c49f16-1552-4a76-a8ab-5db4be497d6c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:51.910115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-00c49f16-1552-4a76-a8ab-5db4be497d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:51.916078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:51.916740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:51.917081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:51.917294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:51.928205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:51.933172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:51.934082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:51.934929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:51.936011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 112/221] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:51.936344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:51.972516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:51.999843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 110/222] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 27 08:19:52.000710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c49f16-1552-4a76-a8ab-5db4be497d6c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.057842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1835f3-ec2d-401b-80da-aad40442b267 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.096632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.097245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.097639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.098454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.107381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:52.131173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c49f16-1552-4a76-a8ab-5db4be497d6c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.169644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.170673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.171105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.171581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.172444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:52.172885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.174685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.182501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.183997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.184906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 111/223] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:52.186053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.223103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:52.313729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1dbeb-ebe5-4c89-9127-7433e6ff9417 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.321112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 113/224] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:52.321965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.356987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.357836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.358274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.358698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.369951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:52.376474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:52.377345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:52.378315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 112/225] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:52.379541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.383325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:52.435352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8670-0c31-4cec-b919-36ac97084200 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.465349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:52.476489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.477317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.477742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.478291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] PATH_INFO: `/v3/users/74f1d05277d147dcb9f686a36e3efbf5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.479734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] RBAC: Authorizing `identity:get_user(user_id=74f1d05277d147dcb9f686a36e3efbf5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.481465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.487902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaba811-20d3-4a40-8018-343aaac945a3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.488148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 114/226] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/users/74f1d05277d147dcb9f686a36e3efbf5 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:52.489624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.526344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.526996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.527279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.527557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.528735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.530213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.531892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 113/227] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:52.532665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.613010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:52.618652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5218ce-ba97-424d-bff2-a018d3b33f78 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.644191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b632849-6dae-435d-8ac0-207373a0cb04 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.655932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.656883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.657308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.657794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.658739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:52.659238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.661592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.670698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 115/228] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:52.671661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.682085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.682681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.682965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.683241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.684403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.685835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.692869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 114/229] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:52.693757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.735559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:52.785761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e8659-57e2-476b-984d-269a895bf488 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.798745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def1b69-9410-4cd8-b8ee-043adf2d5d2e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.824100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.824978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.825405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.825920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.827040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=f97a9cac4cc6489fb979b5c1a95f6139) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:52.827574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.829265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.835447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:52.836088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:52.836373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:52.836474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 116/230] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/role_assignments?user.id=f97a9cac4cc6489fb979b5c1a95f6139&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:52.836644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] PATH_INFO: `/v3/users/74f1d05277d147dcb9f686a36e3efbf5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:52.837485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.843000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] RBAC: Authorizing `identity:get_user(user_id=74f1d05277d147dcb9f686a36e3efbf5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:52.844474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:52.846736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 115/231] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/users/74f1d05277d147dcb9f686a36e3efbf5 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:52.852728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:52.874979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:52.963635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea8f88-5a80-4f88-bec4-2ce5e6e794c8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:52.987481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:53.011674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8c53d-b880-4a4f-9929-24331f050bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:53.012829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8c53d-b880-4a4f-9929-24331f050bb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:53.014835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8c53d-b880-4a4f-9929-24331f050bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:53.015299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8c53d-b880-4a4f-9929-24331f050bb7 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e/users/74f1d05277d147dcb9f686a36e3efbf5/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:53.018889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8c53d-b880-4a4f-9929-24331f050bb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e85f13607084c63b3cb16e16079843e, user_id=74f1d05277d147dcb9f686a36e3efbf5, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:53.022342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8c53d-b880-4a4f-9929-24331f050bb7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:53.050979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1672865-f16a-47a1-8cf2-aef7148d5864 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:53.090806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:53.091433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:53.091730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:53.092046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:53.093285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:53.094934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:53.096985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 116/232] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:52 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:53.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 117/233] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:52 2022] PUT /identity/v3/projects/9e85f13607084c63b3cb16e16079843e/users/74f1d05277d147dcb9f686a36e3efbf5/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:53.104079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:53.194318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:53.220333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877640e5-8b6c-4ed7-8a40-ef47fe321d30 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:53.258788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace401f-8d37-47f5-ae07-7329936eb37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:53.259471 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace401f-8d37-47f5-ae07-7329936eb37e None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:53.259756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace401f-8d37-47f5-ae07-7329936eb37e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:53.260033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace401f-8d37-47f5-ae07-7329936eb37e None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e/users/74f1d05277d147dcb9f686a36e3efbf5/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:53.262465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace401f-8d37-47f5-ae07-7329936eb37e None admin] RBAC: Authorizing `identity:create_grant(project_id=9e85f13607084c63b3cb16e16079843e, user_id=74f1d05277d147dcb9f686a36e3efbf5, role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:53.263965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace401f-8d37-47f5-ae07-7329936eb37e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:53.357127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 117/234] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:53 2022] PUT /identity/v3/projects/9e85f13607084c63b3cb16e16079843e/users/74f1d05277d147dcb9f686a36e3efbf5/roles/d469c6f0fa524f36949e61ae51807884 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:53.873957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef429760-51e6-46fb-aacc-b69eee0e495d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:53.874353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef429760-51e6-46fb-aacc-b69eee0e495d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:53.874622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef429760-51e6-46fb-aacc-b69eee0e495d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:53.876211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 118/235] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:53.881589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ce47-8d9c-4cfb-9c18-afb2b8bea439 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:53.881879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ce47-8d9c-4cfb-9c18-afb2b8bea439 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:53.882180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ce47-8d9c-4cfb-9c18-afb2b8bea439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:53.911840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7165ce47-8d9c-4cfb-9c18-afb2b8bea439 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:53.913062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7165ce47-8d9c-4cfb-9c18-afb2b8bea439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:54.001013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 118/236] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:54.009501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8de9d-03ac-4d53-ba6e-af4e61ae0a6f None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.009786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8de9d-03ac-4d53-ba6e-af4e61ae0a6f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.010057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8de9d-03ac-4d53-ba6e-af4e61ae0a6f None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.017916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 119/237] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:54.018264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4a87-392f-4b78-aabb-408c986f0438 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.018264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4a87-392f-4b78-aabb-408c986f0438 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.018264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef4a87-392f-4b78-aabb-408c986f0438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.033119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8de9d-03ac-4d53-ba6e-af4e61ae0a6f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:54.052915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-39ef4a87-392f-4b78-aabb-408c986f0438 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:54.054052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-39ef4a87-392f-4b78-aabb-408c986f0438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:54.178613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:54.193043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 119/238] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:54.208719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8de9d-03ac-4d53-ba6e-af4e61ae0a6f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:54.227218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4a87-392f-4b78-aabb-408c986f0438 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:54.275251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:54.277007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.278583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.279128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.297654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:54.303832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:54.304755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:54.305541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:54.314000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 120/239] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:54.315885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:54.370527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:54.425988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ff820-c5ae-45ae-a032-8b8181eb0075 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:54.463271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:54.464182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.464655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.465123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.466368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:54.467019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:54.468592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:54.483430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 121/240] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:54.484984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:54.489312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef4a87-392f-4b78-aabb-408c986f0438 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:54.590430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c14da8-fda1-461e-8a1e-34af10a505a8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:54.594846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:54.595578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.595864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.596443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.598590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:54.600050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:54.601672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 120/241] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:54.602118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:54.611102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:54.629931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:54.631232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.631717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.632086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] PATH_INFO: `/v3/users/2fcae17e332643ec8dd2dc2d07726f00` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.646708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] RBAC: Authorizing `identity:get_user(user_id=2fcae17e332643ec8dd2dc2d07726f00)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:54.648476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:54.654303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 122/242] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:54 2022] GET /identity/v3/users/2fcae17e332643ec8dd2dc2d07726f00 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:54.656336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:54.733227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:54.762605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc16c8-09df-4fc9-9bdc-e49a3e4a0b4d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:54.801649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:54.802706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.803200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.803684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.814069 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a6b84-d60b-47ea-839d-238441256749 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:54.834501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:54.876528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:54.878514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:54.899820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:54.900718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:54.901099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:54.901481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:54.902575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:54.904763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:54.991664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 123/243] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 08:19:54.994990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:55.027170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 121/244] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:19:54 2022] POST /identity/v3/users => generated 293 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:55.062488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:55.108355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35d326-27ba-497f-95d6-a8fdee399dcc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:55.149747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439dd95-275f-404a-9a56-225c35ad233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:55.150868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439dd95-275f-404a-9a56-225c35ad233c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.151347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439dd95-275f-404a-9a56-225c35ad233c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.151816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439dd95-275f-404a-9a56-225c35ad233c None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.153344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439dd95-275f-404a-9a56-225c35ad233c None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:55.155199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439dd95-275f-404a-9a56-225c35ad233c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:55.162265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 124/245] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:54 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:55.164670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:55.289767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af11bc-5f39-44d0-8fc0-b16272378a63 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:55.311710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:55.328090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788a2b3-1732-4002-82c9-c0ea8397de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:55.329080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788a2b3-1732-4002-82c9-c0ea8397de2b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.329555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788a2b3-1732-4002-82c9-c0ea8397de2b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.330023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788a2b3-1732-4002-82c9-c0ea8397de2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.331046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788a2b3-1732-4002-82c9-c0ea8397de2b None admin] RBAC: Adding query filter params (role.id=ffc01dcb00834115860b2bf1f7390419, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=2fcae17e332643ec8dd2dc2d07726f00) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:55.331735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788a2b3-1732-4002-82c9-c0ea8397de2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:55.333466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8788a2b3-1732-4002-82c9-c0ea8397de2b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:55.339710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 122/246] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:55 2022] GET /identity/v3/role_assignments?user.id=2fcae17e332643ec8dd2dc2d07726f00&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=ffc01dcb00834115860b2bf1f7390419 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:55.526523 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8463e6-49e1-4c5f-8928-3196fb7f1464 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.527025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8463e6-49e1-4c5f-8928-3196fb7f1464 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.527098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8463e6-49e1-4c5f-8928-3196fb7f1464 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.528688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 125/247] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:55.534792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff50f86-af98-4a3c-b46e-bb003c88c962 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.535980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff50f86-af98-4a3c-b46e-bb003c88c962 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.536461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff50f86-af98-4a3c-b46e-bb003c88c962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.556026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-aff50f86-af98-4a3c-b46e-bb003c88c962 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:55.557548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-aff50f86-af98-4a3c-b46e-bb003c88c962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:55.604212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc3006-815e-4abf-a04b-957556956d3a None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.604519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc3006-815e-4abf-a04b-957556956d3a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.604751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc3006-815e-4abf-a04b-957556956d3a None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.606171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 126/248] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:55.611375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95201-0373-425b-800d-9fe4ffa8710a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.611672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95201-0373-425b-800d-9fe4ffa8710a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.611946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa95201-0373-425b-800d-9fe4ffa8710a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.634460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-caa95201-0373-425b-800d-9fe4ffa8710a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:55.634793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-caa95201-0373-425b-800d-9fe4ffa8710a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:55.637168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 123/249] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:55.661922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff50f86-af98-4a3c-b46e-bb003c88c962 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:55.711512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 127/250] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:55.737401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa95201-0373-425b-800d-9fe4ffa8710a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:55.788874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff50f86-af98-4a3c-b46e-bb003c88c962 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:55.807128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:55.823938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:55.832880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:55.833197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.833472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.833751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.848005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:55.853517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:55.854438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:19:55.855221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:55.860367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 124/251] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:19:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:55.866520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:55.875486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa95201-0373-425b-800d-9fe4ffa8710a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:55.918367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:55.919342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:55.920403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:55.920688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:55.934805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:55.941988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:55.942905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:55.943673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 128/252] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:55.948968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:55.986072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b18e-fc75-4ea6-b732-a1d7f4fe00a2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.022676 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7833d-8138-4092-aa00-94ac0758082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.023292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7833d-8138-4092-aa00-94ac0758082e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.023572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7833d-8138-4092-aa00-94ac0758082e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.023842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7833d-8138-4092-aa00-94ac0758082e None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.024554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7833d-8138-4092-aa00-94ac0758082e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:56.024852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7833d-8138-4092-aa00-94ac0758082e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:56.026255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7833d-8138-4092-aa00-94ac0758082e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:56.037358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 125/253] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:56.038821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b358-bce3-41d7-bb95-46694f141782 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.039111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b358-bce3-41d7-bb95-46694f141782 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.039384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b358-bce3-41d7-bb95-46694f141782 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.041499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 126/254] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:56.047388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ed422-b202-4d88-bfad-164ddfe5191b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.047672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ed422-b202-4d88-bfad-164ddfe5191b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.047946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ed422-b202-4d88-bfad-164ddfe5191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.066277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cd9ed422-b202-4d88-bfad-164ddfe5191b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:56.067351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cd9ed422-b202-4d88-bfad-164ddfe5191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:56.079955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e4ff4-b315-4de3-9617-cc21eff5bd77 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.099086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:56.104575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:56.140246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.140872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.141156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.141430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.142180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:56.142477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:56.143881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:56.162556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 129/255] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:56.164861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 127/256] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:56.165463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:56.171371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ed422-b202-4d88-bfad-164ddfe5191b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:56.343995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ed422-b202-4d88-bfad-164ddfe5191b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.361948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4b93f-3446-4392-9d95-1f7cca67f3da None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.383489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.384763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.385115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.385482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] PATH_INFO: `/v3/users/f97a9cac4cc6489fb979b5c1a95f6139` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.387236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] RBAC: Authorizing `identity:get_user(user_id=f97a9cac4cc6489fb979b5c1a95f6139)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:56.390368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:56.390368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 128/257] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:56 2022] GET /identity/v3/users/f97a9cac4cc6489fb979b5c1a95f6139 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:56.392328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:56.440727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.442084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.442393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.443319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] PATH_INFO: `/v3/users/74f1d05277d147dcb9f686a36e3efbf5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.443991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] RBAC: Authorizing `identity:get_user(user_id=74f1d05277d147dcb9f686a36e3efbf5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:56.445447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:56.449302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 130/258] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:56 2022] GET /identity/v3/users/74f1d05277d147dcb9f686a36e3efbf5 => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:56.450695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:56.453773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:56.546451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:56.583739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d46ae4-8e26-4230-844c-213390f72773 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.621177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba24823-4717-49d4-aacb-bb5c95de67ed None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.643622 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.644772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.645044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.645300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.659706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.661294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.661823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.662339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.663820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:56.665503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:56.667485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 131/259] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:56 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:56.668485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:56.669085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:56.680663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:56.681682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:56.682952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 129/260] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:19:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:19:56.687005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:56.777771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa81f41-ed64-4c35-bc37-91f0ba487544 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.806564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:56.822601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.823591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.824136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.824739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.845121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:56.850596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beef0d-6591-4175-a8a8-130ad53de6dd None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:56.920359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:56.920951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:56.921230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:56.921503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e/users/f97a9cac4cc6489fb979b5c1a95f6139/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:56.926754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e85f13607084c63b3cb16e16079843e, user_id=f97a9cac4cc6489fb979b5c1a95f6139, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:56.928269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:56.947874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:56.950949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:56.996469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 130/261] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:56 2022] PUT /identity/v3/projects/9e85f13607084c63b3cb16e16079843e/users/f97a9cac4cc6489fb979b5c1a95f6139/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:56.997420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:57.129688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 132/262] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:19:57.137081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:57.169806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:57.200880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344640c-cb3a-4191-84b5-304cfca24b62 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:57.250163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:57.250816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:57.251094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:57.251367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:57.252080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:57.252433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:57.253856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:57.263875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 131/263] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:19:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:57.269647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:57.281366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315fcad-311e-43d6-9844-e0c29e5e7652 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:57.297875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:57.326685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:57.327327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:57.327609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:57.327885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:57.329563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:57.331435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:57.334766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 133/264] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:57 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:57.350183 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:57.427940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5a461-ac71-428c-acab-18f8e1311d6f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:57.461119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:57.478204 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96201dd-fc81-4222-b899-9a42b3d26af8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:57.487099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:57.488260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:57.488733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:57.489043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] PATH_INFO: `/v3/users/74f1d05277d147dcb9f686a36e3efbf5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:57.492473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] RBAC: Authorizing `identity:get_user(user_id=74f1d05277d147dcb9f686a36e3efbf5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:57.493923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:57.496543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 132/265] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:57 2022] GET /identity/v3/users/74f1d05277d147dcb9f686a36e3efbf5 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:19:57.500933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:57.522000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:57.523018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:57.524836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:57.525314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:57.526999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=74f1d05277d147dcb9f686a36e3efbf5) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:57.527712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:57.530539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:57.542358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 134/266] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:57 2022] GET /identity/v3/role_assignments?user.id=74f1d05277d147dcb9f686a36e3efbf5&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:57.642102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:57.663426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22290c48-96c3-47bb-b737-e17a16348868 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:57.724665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074ea72-ec40-49f7-af62-6344268ddb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:57.725219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ea72-ec40-49f7-af62-6344268ddb7b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:57.725505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ea72-ec40-49f7-af62-6344268ddb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:57.725783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ea72-ec40-49f7-af62-6344268ddb7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:57.818727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074ea72-ec40-49f7-af62-6344268ddb7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:57.820236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074ea72-ec40-49f7-af62-6344268ddb7b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:57.941970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:57.960236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 133/267] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:19:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 08:19:57.965441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:58.039202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e8ac-b7c3-4a5a-bf66-e01a50bb7c46 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.039697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e8ac-b7c3-4a5a-bf66-e01a50bb7c46 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.040187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714e8ac-b7c3-4a5a-bf66-e01a50bb7c46 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.042017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 134/268] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:58.047317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad9c5-6759-420e-974f-c53fc373041a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.047867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad9c5-6759-420e-974f-c53fc373041a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.048345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad9c5-6759-420e-974f-c53fc373041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.091222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ed0ad9c5-6759-420e-974f-c53fc373041a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:58.092807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ed0ad9c5-6759-420e-974f-c53fc373041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:58.099572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9380e3-4b90-4c4b-b72e-f29e40bd028d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:58.136826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:58.137470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.137795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.138070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.141048 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:58.142613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:58.144653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 135/269] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:57 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:58.150427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:58.172867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:58.278727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 135/270] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:58.292518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df277b-7878-4aca-a40d-3a0962435666 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:58.300528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad9c5-6759-420e-974f-c53fc373041a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:58.375403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9419d4-8356-4f15-81de-8ce7062574c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:58.376089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9419d4-8356-4f15-81de-8ce7062574c8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.376367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9419d4-8356-4f15-81de-8ce7062574c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.376635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9419d4-8356-4f15-81de-8ce7062574c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.377433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9419d4-8356-4f15-81de-8ce7062574c8 None admin] RBAC: Adding query filter params (role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=74f1d05277d147dcb9f686a36e3efbf5) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:58.377890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9419d4-8356-4f15-81de-8ce7062574c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:58.379616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9419d4-8356-4f15-81de-8ce7062574c8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:58.397039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 136/271] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:19:58 2022] GET /identity/v3/role_assignments?user.id=74f1d05277d147dcb9f686a36e3efbf5&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=d469c6f0fa524f36949e61ae51807884 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:58.449002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b06d6d-2716-4781-a941-e95d2a6bff8e None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.449325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b06d6d-2716-4781-a941-e95d2a6bff8e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.449579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b06d6d-2716-4781-a941-e95d2a6bff8e None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.451335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 137/272] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:58.459681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4289a-ed02-46a8-893e-67c30e28026e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.459903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad9c5-6759-420e-974f-c53fc373041a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:58.460188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4289a-ed02-46a8-893e-67c30e28026e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.460625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4289a-ed02-46a8-893e-67c30e28026e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.480006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-48a4289a-ed02-46a8-893e-67c30e28026e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:58.481372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-48a4289a-ed02-46a8-893e-67c30e28026e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:58.489594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:58.544092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:58.545089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.545563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.546025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.572153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:58.577577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:58.578590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:19:58.579391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:58.582349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 136/273] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:58.586632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:58.599340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 138/274] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:58.625950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4289a-ed02-46a8-893e-67c30e28026e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:58.640751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:58.736681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e62a0-f859-415d-a0f8-b1930f63dafa None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:58.764069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4289a-ed02-46a8-893e-67c30e28026e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:58.792910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643228-747d-4208-ae8c-92484502960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:58.793525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643228-747d-4208-ae8c-92484502960a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.793804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643228-747d-4208-ae8c-92484502960a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.794072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643228-747d-4208-ae8c-92484502960a None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.794901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643228-747d-4208-ae8c-92484502960a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:58.795144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643228-747d-4208-ae8c-92484502960a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:58.796521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643228-747d-4208-ae8c-92484502960a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:58.808179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 137/275] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:58.809520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c9c8-95e7-48f9-892b-68b6c48d3b4f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.809643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb485d9-10bc-4559-886e-0ac87e3275cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:58.809865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c9c8-95e7-48f9-892b-68b6c48d3b4f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.810079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c9c8-95e7-48f9-892b-68b6c48d3b4f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.810282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb485d9-10bc-4559-886e-0ac87e3275cf None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.810576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb485d9-10bc-4559-886e-0ac87e3275cf None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.810853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb485d9-10bc-4559-886e-0ac87e3275cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.811310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 138/276] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:58.816699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4c9c8-95e7-48f9-892b-68b6c48d3b4f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:58.822607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb485d9-10bc-4559-886e-0ac87e3275cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-cfb485d9-10bc-4559-886e-0ac87e3275cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:58.827572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:58.828434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:19:58.829260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 139/277] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:58.834992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e3ce-0644-4071-b783-6e01a29a2e24 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.834992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e3ce-0644-4071-b783-6e01a29a2e24 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.834992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e3ce-0644-4071-b783-6e01a29a2e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.850902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-69f7e3ce-0644-4071-b783-6e01a29a2e24 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:58.852073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-69f7e3ce-0644-4071-b783-6e01a29a2e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:58.860951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:58.922297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:58.927589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 140/278] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:19:58.928426 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7e3ce-0644-4071-b783-6e01a29a2e24 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:58.945244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4c9c8-95e7-48f9-892b-68b6c48d3b4f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:58.983605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:58.984251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:58.984882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:58.985177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] PATH_INFO: `/v3/users/2fcae17e332643ec8dd2dc2d07726f00` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:58.986894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] RBAC: Authorizing `identity:get_user(user_id=2fcae17e332643ec8dd2dc2d07726f00)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:58.988809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:58.994819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 139/279] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:58 2022] GET /identity/v3/users/2fcae17e332643ec8dd2dc2d07726f00 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:58.994819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.052199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7e3ce-0644-4071-b783-6e01a29a2e24 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.087699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.088335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.088766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.089047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.089779 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:59.090087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:59.091518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:59.099746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefc0ba-c230-4674-80dc-75afb11db493 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.101217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 141/280] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:59.105805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.106247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:59.141272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.141916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.142218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.142496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.143645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:59.145935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:59.150334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 140/281] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:59.151200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.176535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:59.217997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba20ef-81c8-43b9-9689-8cc7e3fe10d1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.255246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4794b-57dc-4d2f-89d1-0cf212a199fe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.260394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.261079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.261385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.261664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.263277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:59.265090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:59.267103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 142/282] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:19:59 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:59.272075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.297146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.297795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.298076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.298393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] PATH_INFO: `/v3/users/4b6aececa6af420db9e2853914205de2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.313147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] RBAC: Authorizing `identity:get_user(user_id=4b6aececa6af420db9e2853914205de2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:59.314592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:59.316556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 141/283] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:19:59 2022] GET /identity/v3/users/4b6aececa6af420db9e2853914205de2 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:59.317298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.352110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:59.386645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b552-bfff-485c-8355-d4ae127158e9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.418241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:59.424596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.425818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.426105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.426441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.431207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:59.431207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:59.431382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8faba-5f0d-4959-a561-6dfe7b07eee7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.465902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5237306-2b52-4882-9ffa-67147c8ca264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.466621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237306-2b52-4882-9ffa-67147c8ca264 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.466898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237306-2b52-4882-9ffa-67147c8ca264 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.467166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237306-2b52-4882-9ffa-67147c8ca264 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e/users/2fcae17e332643ec8dd2dc2d07726f00/roles/ffc01dcb00834115860b2bf1f7390419` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.469179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5237306-2b52-4882-9ffa-67147c8ca264 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e85f13607084c63b3cb16e16079843e, user_id=2fcae17e332643ec8dd2dc2d07726f00, role_id=ffc01dcb00834115860b2bf1f7390419)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:59.470651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5237306-2b52-4882-9ffa-67147c8ca264 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:59.498634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 143/284] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:19:59 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:19:59.511302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.553765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 142/285] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:19:59 2022] PUT /identity/v3/projects/9e85f13607084c63b3cb16e16079843e/users/2fcae17e332643ec8dd2dc2d07726f00/roles/ffc01dcb00834115860b2bf1f7390419 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:19:59.553765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf2e94-1251-4a00-a60f-2aa1b1836201 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.553765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf2e94-1251-4a00-a60f-2aa1b1836201 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.553765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf2e94-1251-4a00-a60f-2aa1b1836201 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.553765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 143/286] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:19:59.564067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed490e8-3dcc-42a8-a61c-f3fc0142ca7a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.564281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed490e8-3dcc-42a8-a61c-f3fc0142ca7a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.564555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed490e8-3dcc-42a8-a61c-f3fc0142ca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.591458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6ed490e8-3dcc-42a8-a61c-f3fc0142ca7a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:19:59.593136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6ed490e8-3dcc-42a8-a61c-f3fc0142ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:19:59.655323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e03ca-1dfa-434d-9bd3-3dc76a0bb9c7 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.676481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 144/287] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:19:59.698597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.700012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.700558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed490e8-3dcc-42a8-a61c-f3fc0142ca7a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.700669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.701173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.702217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, user.id=4b6aececa6af420db9e2853914205de2, scope.system=all) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:19:59.702851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:19:59.704629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:19:59.711506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 144/288] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 27 08:19:59 2022] GET /identity/v3/role_assignments?user.id=4b6aececa6af420db9e2853914205de2&scope.system=all&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:19:59.727894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:19:59.823457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed490e8-3dcc-42a8-a61c-f3fc0142ca7a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:19:59.867633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:19:59.868294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:19:59.868572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:19:59.868844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:19:59.881145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:19:59.886214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:19:59.887122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:19:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:19:59.890889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 145/289] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:19:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:19:59.899206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:19:59.978505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:00.012254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50032fb2-23e9-4f4e-87ee-c251c3671c79 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:00.053041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:00.053675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:00.053933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:00.054237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:00.054961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:00.055263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:00.056645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:00.072379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 145/290] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:19:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:00.079314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:00.210533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:00.235592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a3832-3cc6-4fa7-a877-60cf1517064e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:00.280953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:00.281602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:00.281884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:00.282170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] PATH_INFO: `/v3/users/f97a9cac4cc6489fb979b5c1a95f6139` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:00.283359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] RBAC: Authorizing `identity:get_user(user_id=f97a9cac4cc6489fb979b5c1a95f6139)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:00.285382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:00.287361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 146/291] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:20:00 2022] GET /identity/v3/users/f97a9cac4cc6489fb979b5c1a95f6139 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:00.292842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:00.453993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8774f80-720e-41c9-aa53-57407250a7a0 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:00.472575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:00.503901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:00.504863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:00.505376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:00.505850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:00.609709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:00.611208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:00.735231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 146/292] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:00.743232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:00.877768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e32cd-bda5-47e2-b055-40c9be6994ed None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:00.923672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70e09d-360c-4ae7-8b6f-e64a0b15241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:00.924311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70e09d-360c-4ae7-8b6f-e64a0b15241a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:00.924589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70e09d-360c-4ae7-8b6f-e64a0b15241a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:00.924862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70e09d-360c-4ae7-8b6f-e64a0b15241a None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:00.926221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70e09d-360c-4ae7-8b6f-e64a0b15241a None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:00.927662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70e09d-360c-4ae7-8b6f-e64a0b15241a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:00.929489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 147/293] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:00 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:00.935306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:00.954491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:01.053565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51800ce2-d8f0-4356-a1e3-cc19a940d1d7 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:01.103827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3de041-494f-4185-8908-1f68aa59f579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:01.104811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3de041-494f-4185-8908-1f68aa59f579 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:01.105318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3de041-494f-4185-8908-1f68aa59f579 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:01.105791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3de041-494f-4185-8908-1f68aa59f579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:01.106826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3de041-494f-4185-8908-1f68aa59f579 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=f97a9cac4cc6489fb979b5c1a95f6139) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:01.107339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3de041-494f-4185-8908-1f68aa59f579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:01.108994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3de041-494f-4185-8908-1f68aa59f579 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:01.118611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 147/294] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:00 2022] GET /identity/v3/role_assignments?user.id=f97a9cac4cc6489fb979b5c1a95f6139&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:02.002066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58406788-d621-48da-9190-650f89073f17 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.002604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58406788-d621-48da-9190-650f89073f17 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.002604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58406788-d621-48da-9190-650f89073f17 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.004197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 148/295] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:02.013517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd856b12-3518-4661-bf05-27b55210d8bd None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.013841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd856b12-3518-4661-bf05-27b55210d8bd None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.014122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd856b12-3518-4661-bf05-27b55210d8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.032973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-cd856b12-3518-4661-bf05-27b55210d8bd None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:02.034172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cd856b12-3518-4661-bf05-27b55210d8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:02.131969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 148/296] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:02.163266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58406788-d621-48da-9190-650f89073f17 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.198041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.217462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7f2e-2c5b-4196-8dd7-557ab3a24554 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.217743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7f2e-2c5b-4196-8dd7-557ab3a24554 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.218241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7f2e-2c5b-4196-8dd7-557ab3a24554 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.219787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 149/297] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:02.225325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f401-853b-4ca2-903c-8b687bbb94c7 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.225325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f401-853b-4ca2-903c-8b687bbb94c7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.225325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492f401-853b-4ca2-903c-8b687bbb94c7 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.226849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 150/298] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:02.227771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56624815-6860-4613-8741-50aa4d6d8ac5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.228057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56624815-6860-4613-8741-50aa4d6d8ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.228407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56624815-6860-4613-8741-50aa4d6d8ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.252254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.258610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-56624815-6860-4613-8741-50aa4d6d8ac5 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:02.259948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-56624815-6860-4613-8741-50aa4d6d8ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:02.309760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58406788-d621-48da-9190-650f89073f17 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:02.327193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 151/299] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:02.328436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d3f0-3816-4a83-8b43-d6cd5a03c938 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.328724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d3f0-3816-4a83-8b43-d6cd5a03c938 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.328996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d3f0-3816-4a83-8b43-d6cd5a03c938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.347980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1ae4d3f0-3816-4a83-8b43-d6cd5a03c938 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:02.349355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1ae4d3f0-3816-4a83-8b43-d6cd5a03c938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:02.349912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e6328-c13f-4670-b101-31bcfa420e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:02.350576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e6328-c13f-4670-b101-31bcfa420e0b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.350856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e6328-c13f-4670-b101-31bcfa420e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.351128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e6328-c13f-4670-b101-31bcfa420e0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.361556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e6328-c13f-4670-b101-31bcfa420e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-537e6328-c13f-4670-b101-31bcfa420e0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:02.366774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:02.367647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:02.368711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 149/300] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:02.369595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71186bdf-f5f0-4797-bca6-e68bd97290d5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.369595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71186bdf-f5f0-4797-bca6-e68bd97290d5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.369734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71186bdf-f5f0-4797-bca6-e68bd97290d5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.371026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 150/301] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:02.375475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71186bdf-f5f0-4797-bca6-e68bd97290d5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.426717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 152/302] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:02.427550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4d3f0-3816-4a83-8b43-d6cd5a03c938 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.457413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.480894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71186bdf-f5f0-4797-bca6-e68bd97290d5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:02.492503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.515250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0b143-e250-4ed4-9d27-e2204f897f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:02.515820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0b143-e250-4ed4-9d27-e2204f897f2a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.516100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0b143-e250-4ed4-9d27-e2204f897f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.516439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0b143-e250-4ed4-9d27-e2204f897f2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.526277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4d3f0-3816-4a83-8b43-d6cd5a03c938 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:02.526658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0b143-e250-4ed4-9d27-e2204f897f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-78d0b143-e250-4ed4-9d27-e2204f897f2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:02.532002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:02.532876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:02.533675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 151/303] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:02.534893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deb246-6643-4554-a456-1590d5de69dd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.535438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deb246-6643-4554-a456-1590d5de69dd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.535632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deb246-6643-4554-a456-1590d5de69dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.552953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-08deb246-6643-4554-a456-1590d5de69dd None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:02.554200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-08deb246-6643-4554-a456-1590d5de69dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:02.565495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:02.566832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.567153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.567383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.568265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:02.569476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:02.570352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:02.578455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 153/304] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:02.579239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.621520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 152/305] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:02.622608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deb246-6643-4554-a456-1590d5de69dd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.682581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f5a50-d84f-4ce8-9f0e-029508dcbfd2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:02.699087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.708501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.716251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:02.716884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.717163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.717435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.728123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:02.733424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:02.740300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:02.741124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:02.741720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deb246-6643-4554-a456-1590d5de69dd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:02.741798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 154/306] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:02.741798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.769576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:02.770231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.770507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.770773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.771472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:02.771769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:02.773135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:02.780914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 153/307] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:02.781685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.832214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577615f-911e-4927-9538-6a17a399eb17 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:02.865927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:02.866584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.866868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.867146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] PATH_INFO: `/v3/users/2fcae17e332643ec8dd2dc2d07726f00` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.868311 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] RBAC: Authorizing `identity:get_user(user_id=2fcae17e332643ec8dd2dc2d07726f00)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:02.869771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:02.871396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 155/308] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/users/2fcae17e332643ec8dd2dc2d07726f00 => generated 297 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:02.874998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.879448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97f5c8-93ee-4ea1-87bb-b34beeed6487 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:02.905607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.914092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:02.914767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:02.915042 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:02.918211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:02.918211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:02.918211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:02.919648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 154/309] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:02.924334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:02.932819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:02.976685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6cc7a-5f87-4a31-8ef3-bcf5703aa09d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.008908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.009517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.009844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.010275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.010984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:03.011285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.012645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.017722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbad20-6c2c-434e-b2d8-6787b368f6d4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.021234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 156/310] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:03.026161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:03.051827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.052465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.052744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.053034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] PATH_INFO: `/v3/users/4b6aececa6af420db9e2853914205de2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.054209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] RBAC: Authorizing `identity:get_user(user_id=4b6aececa6af420db9e2853914205de2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.055645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.057219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 155/311] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:20:02 2022] GET /identity/v3/users/4b6aececa6af420db9e2853914205de2 => generated 293 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:03.057953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:03.121139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:03.127192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c2a8-a352-4d3d-8e07-9ed722c2c2f9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.145573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:03.157368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcbe14-8d46-4978-8452-9bbca5f5ffa7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.162726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.163341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.163638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.163894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.191599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.192234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.192508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.192775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] PATH_INFO: `/v3/groups` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.193420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.194897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.233665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.235151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.262112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 156/312] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Jul 27 08:20:03 2022] POST /identity/v3/groups => generated 229 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:03.262112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:03.351972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 157/313] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:03.351972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:03.370358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:03.381401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4107e-6dd1-4606-a720-75e2dfec1fbb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.409056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:03.424015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.424669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.425436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.425709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] PATH_INFO: `/v3/users/0c2e67a074e84e1ba5003585aff31e0e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.454268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] RBAC: Authorizing `identity:get_user(user_id=0c2e67a074e84e1ba5003585aff31e0e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.455721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.458462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 157/314] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:20:03 2022] GET /identity/v3/users/0c2e67a074e84e1ba5003585aff31e0e => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:03.459015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:03.469878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bdd0d-5230-459e-a3a3-e4a238eb6c11 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.509312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.509962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.510266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.510535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] PATH_INFO: `/v3/system/users/4b6aececa6af420db9e2853914205de2/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.511976 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4b6aececa6af420db9e2853914205de2, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.513353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.564501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4b26-bb42-485b-9964-8eb151364b34 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.596472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 158/315] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Jul 27 08:20:03 2022] PUT /identity/v3/system/users/4b6aececa6af420db9e2853914205de2/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:03.598352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:03.609796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.611800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.612303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.612884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.614661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.617048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.619730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 158/316] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:03 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:03.627280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:03.707145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:03.709317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:03.729821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9318e69-fb32-48bb-9dff-a19cd6a112e6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.775743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d239295-57bd-4b9d-9357-d417a5652c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.776393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d239295-57bd-4b9d-9357-d417a5652c7d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.776668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d239295-57bd-4b9d-9357-d417a5652c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.776936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d239295-57bd-4b9d-9357-d417a5652c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.777455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215901-7832-4ba4-b413-3380419160a1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:03.777728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d239295-57bd-4b9d-9357-d417a5652c7d None admin] RBAC: Adding query filter params (role.id=ffc01dcb00834115860b2bf1f7390419, user.id=0c2e67a074e84e1ba5003585aff31e0e, scope.system=all) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:03.778037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d239295-57bd-4b9d-9357-d417a5652c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.779532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d239295-57bd-4b9d-9357-d417a5652c7d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.785565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 159/317] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 27 08:20:03 2022] GET /identity/v3/role_assignments?user.id=0c2e67a074e84e1ba5003585aff31e0e&scope.system=all&role.id=ffc01dcb00834115860b2bf1f7390419 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:03.821143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacecfa-7016-4a7d-860c-f314d8b63791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:03.821784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacecfa-7016-4a7d-860c-f314d8b63791 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:03.822070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacecfa-7016-4a7d-860c-f314d8b63791 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:03.822399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacecfa-7016-4a7d-860c-f314d8b63791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:03.823216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacecfa-7016-4a7d-860c-f314d8b63791 None admin] RBAC: Adding query filter params (role.id=ffc01dcb00834115860b2bf1f7390419, scope.project.id=9e85f13607084c63b3cb16e16079843e, user.id=2fcae17e332643ec8dd2dc2d07726f00) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:03.823532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacecfa-7016-4a7d-860c-f314d8b63791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:03.825130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacecfa-7016-4a7d-860c-f314d8b63791 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:03.835485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 159/318] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:03 2022] GET /identity/v3/role_assignments?user.id=2fcae17e332643ec8dd2dc2d07726f00&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=ffc01dcb00834115860b2bf1f7390419 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:05.324009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16409208-3da8-4f86-a3d8-5a5503e04aed None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.324570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16409208-3da8-4f86-a3d8-5a5503e04aed None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.324570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16409208-3da8-4f86-a3d8-5a5503e04aed None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.326086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 160/319] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:05.330995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2837e-982c-4c7a-8e49-36b01db506da None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.332031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2837e-982c-4c7a-8e49-36b01db506da None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.332105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2837e-982c-4c7a-8e49-36b01db506da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.350456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-49b2837e-982c-4c7a-8e49-36b01db506da None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:05.351647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-49b2837e-982c-4c7a-8e49-36b01db506da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:05.419955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 160/320] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:05.441740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16409208-3da8-4f86-a3d8-5a5503e04aed None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:05.542333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16409208-3da8-4f86-a3d8-5a5503e04aed None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:05.581790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0ef29-2bf2-4dc9-bda9-0bed760a2710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:05.582452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ef29-2bf2-4dc9-bda9-0bed760a2710 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.582728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ef29-2bf2-4dc9-bda9-0bed760a2710 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.583000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ef29-2bf2-4dc9-bda9-0bed760a2710 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.584131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0ef29-2bf2-4dc9-bda9-0bed760a2710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:05.585579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0ef29-2bf2-4dc9-bda9-0bed760a2710 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:05.587384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 161/321] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:05.592680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2837e-982c-4c7a-8e49-36b01db506da None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:05.638203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:05.642520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546843c5-33d5-4544-9703-a83e1e500e30 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.642804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546843c5-33d5-4544-9703-a83e1e500e30 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.643251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546843c5-33d5-4544-9703-a83e1e500e30 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.645197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 162/322] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:20:05.650581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8990ab-7106-410c-b968-bded037278b8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.650866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8990ab-7106-410c-b968-bded037278b8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.651138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8990ab-7106-410c-b968-bded037278b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.669685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6b8990ab-7106-410c-b968-bded037278b8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:05.670934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6b8990ab-7106-410c-b968-bded037278b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:05.680698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:05.694433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2837e-982c-4c7a-8e49-36b01db506da None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:05.739318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158a02a-2ed4-4af3-ab19-74bfed1d199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:05.740296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158a02a-2ed4-4af3-ab19-74bfed1d199b None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.740578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158a02a-2ed4-4af3-ab19-74bfed1d199b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.740841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158a02a-2ed4-4af3-ab19-74bfed1d199b None admin] PATH_INFO: `/v3/groups` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.741519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158a02a-2ed4-4af3-ab19-74bfed1d199b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:05.743141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158a02a-2ed4-4af3-ab19-74bfed1d199b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:05.750376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 163/323] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:20:05.773842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8990ab-7106-410c-b968-bded037278b8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:05.818641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 161/324] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Jul 27 08:20:05 2022] POST /identity/v3/groups => generated 226 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:20:05.819897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d8493-73d3-4aa4-a812-3fc95695ab8c None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.820190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d8493-73d3-4aa4-a812-3fc95695ab8c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.820470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d8493-73d3-4aa4-a812-3fc95695ab8c None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.821677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 162/325] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:05.827393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5db225-6c89-4b63-be0c-b50edb6e412a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.827678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5db225-6c89-4b63-be0c-b50edb6e412a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.828121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5db225-6c89-4b63-be0c-b50edb6e412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.848178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-fc5db225-6c89-4b63-be0c-b50edb6e412a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:05.849403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fc5db225-6c89-4b63-be0c-b50edb6e412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:05.885340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8990ab-7106-410c-b968-bded037278b8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:05.919302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 163/326] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:20:05.924561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:05.925265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:05.925550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:05.925825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:05.937217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:05.941444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5db225-6c89-4b63-be0c-b50edb6e412a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:05.942839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:05.943719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:05.944561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 164/327] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:05.950135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:05.954542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:05.981026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:06.062517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5db225-6c89-4b63-be0c-b50edb6e412a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:06.073450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d878fb-fa45-4619-9c1d-9f6cf90d0979 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:06.116734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:06.117310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:06.117588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:06.117858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:06.122474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:06.123238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:06.123524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:06.123793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:06.124661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:06.124965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:06.126807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:06.131146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:06.136288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:06.137162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:06.137956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:06.138716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 165/328] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:06.138788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 164/329] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:06.144305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:06.147017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:06.233266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:06.323212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:06.331781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f47a5d-caaa-48b0-9389-c3dca7e0f3a1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:06.332663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1aed7-757c-4c41-b521-650594dcb32f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:06.372149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:06.372844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:06.373127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:06.373398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:06.374224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:06.374536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:06.375591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:06.375927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:06.376229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:06.376505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:06.376769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] PATH_INFO: `/v3/users/4b6aececa6af420db9e2853914205de2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:06.378563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] RBAC: Authorizing `identity:get_user(user_id=4b6aececa6af420db9e2853914205de2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:06.388213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:06.388213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 165/330] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:20:06 2022] GET /identity/v3/users/4b6aececa6af420db9e2853914205de2 => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:06.388601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 166/331] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:06.390193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:06.391129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:06.526546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b76c0-a969-41ab-b40b-86b2a05dc832 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:06.529142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f7579-fc54-431b-a2e1-62782caf56fa None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:06.557665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:06.567924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:06.568107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1592cea-0e59-4be9-977d-cf803d777728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:06.568552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1592cea-0e59-4be9-977d-cf803d777728 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:06.568825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1592cea-0e59-4be9-977d-cf803d777728 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:06.577600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1592cea-0e59-4be9-977d-cf803d777728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:06.578469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:06.578469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:06.578469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] PATH_INFO: `/v3/users/0c2e67a074e84e1ba5003585aff31e0e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:06.578668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1592cea-0e59-4be9-977d-cf803d777728 None admin] RBAC: Adding query filter params (role.id=6a60961045ef46cd91a802e9a6cf7b9e, user.id=4b6aececa6af420db9e2853914205de2, scope.system=all) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:06.578668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1592cea-0e59-4be9-977d-cf803d777728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:06.578809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] RBAC: Authorizing `identity:get_user(user_id=0c2e67a074e84e1ba5003585aff31e0e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:06.578874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1592cea-0e59-4be9-977d-cf803d777728 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:06.578938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:06.578938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 167/332] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:20:06 2022] GET /identity/v3/users/0c2e67a074e84e1ba5003585aff31e0e => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:06.582525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:06.586053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 166/333] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 27 08:20:06 2022] GET /identity/v3/role_assignments?user.id=4b6aececa6af420db9e2853914205de2&scope.system=all&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:06.718629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4db61-cd48-4dda-b9ba-5f39b849231c None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:06.763445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14257408-302a-4d49-a15e-3f2493048b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:06.764118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14257408-302a-4d49-a15e-3f2493048b1b None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:06.764395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14257408-302a-4d49-a15e-3f2493048b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:06.764668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14257408-302a-4d49-a15e-3f2493048b1b None admin] PATH_INFO: `/v3/system/users/0c2e67a074e84e1ba5003585aff31e0e/roles/ffc01dcb00834115860b2bf1f7390419` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:06.766134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14257408-302a-4d49-a15e-3f2493048b1b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0c2e67a074e84e1ba5003585aff31e0e, role_id=ffc01dcb00834115860b2bf1f7390419)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:06.767542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14257408-302a-4d49-a15e-3f2493048b1b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:06.864925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 168/334] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Jul 27 08:20:06 2022] PUT /identity/v3/system/users/0c2e67a074e84e1ba5003585aff31e0e/roles/ffc01dcb00834115860b2bf1f7390419 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:08.499023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d629c-f40f-4adf-b5a3-d37f6b54b46d None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:08.499535 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d629c-f40f-4adf-b5a3-d37f6b54b46d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:08.499610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d629c-f40f-4adf-b5a3-d37f6b54b46d None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:08.501179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 167/335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:08.507739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703aa1be-12ee-4cef-ac9b-d6f064f59d31 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:08.508183 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703aa1be-12ee-4cef-ac9b-d6f064f59d31 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:08.508365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703aa1be-12ee-4cef-ac9b-d6f064f59d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:08.530812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-703aa1be-12ee-4cef-ac9b-d6f064f59d31 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:08.531998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-703aa1be-12ee-4cef-ac9b-d6f064f59d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:08.617681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:08.632391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 169/336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:08.675916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d629c-f40f-4adf-b5a3-d37f6b54b46d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:08.708954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:08.812758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d629c-f40f-4adf-b5a3-d37f6b54b46d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:08.826725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825a9cb-a3b5-4c0b-95cc-3d5054c7c7bd None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:08.827032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825a9cb-a3b5-4c0b-95cc-3d5054c7c7bd None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:08.827281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825a9cb-a3b5-4c0b-95cc-3d5054c7c7bd None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:08.828897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 170/337] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:08.835606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870d2b-947a-4b4c-b796-a1e6cd4dde18 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:08.835892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870d2b-947a-4b4c-b796-a1e6cd4dde18 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:08.836168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870d2b-947a-4b4c-b796-a1e6cd4dde18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:08.866267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:08.866930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:08.867217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:08.867481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:08.874261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e2870d2b-947a-4b4c-b796-a1e6cd4dde18 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:08.875468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e2870d2b-947a-4b4c-b796-a1e6cd4dde18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:08.887524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:08.893738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:08.894673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:08.895462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:08.899039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 168/338] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:08.905213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.011823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 171/339] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:09.022747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:09.040071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b15a3-40dc-4204-bda7-ca5be288412f None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.040558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b15a3-40dc-4204-bda7-ca5be288412f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.040812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b15a3-40dc-4204-bda7-ca5be288412f None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.042182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 172/340] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:20:09.046506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba25b6-d0fe-4091-b134-d5cc074bc81b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:09.052411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c7a47-f7f4-4fd5-826a-578766fb2099 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.052699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c7a47-f7f4-4fd5-826a-578766fb2099 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.052971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c7a47-f7f4-4fd5-826a-578766fb2099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.073082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4c7c7a47-f7f4-4fd5-826a-578766fb2099 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:09.074222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4c7c7a47-f7f4-4fd5-826a-578766fb2099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:09.098881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:09.099521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.099800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.100066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.100785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:09.101084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:09.102471 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:09.111608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 169/341] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:09.115044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.151936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 173/342] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:09.152800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c7a47-f7f4-4fd5-826a-578766fb2099 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.224668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:09.256953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49def86e-d4fb-42ca-99c0-d1f3d8411154 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:09.260158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c7a47-f7f4-4fd5-826a-578766fb2099 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:09.310778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:09.311427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.311709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.311981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.323694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:09.329470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:09.330459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 170/343] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:20:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:09.332452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.345086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:09.358887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:09.359539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.365022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.365022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.446740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4256457-751d-4c19-b7a8-db0bd9a48cec None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:09.487119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:09.487974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.489178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.489474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.502087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:09.509342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:09.510285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:09.511182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 171/344] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:09.512367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.586278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:09.596078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:09.597632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:09.631841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1250e13-9161-4e78-a243-aa91a30148a8 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:09.675064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:09.675694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.675971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.676241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.678481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:09.678803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:09.681108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:09.686781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:09.700267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 172/345] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:09.701088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.790480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 174/346] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:09.791142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.902451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be580c75-824b-4b9a-b297-4505ee757f76 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:09.931773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:09.947163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:09.947792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:09.948071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:09.948342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:09.955002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:09.955002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:09.955002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:09.986111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 173/347] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:09.991318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:09.994603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82c611-ecac-4427-9468-698073fdc685 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:10.073343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814127a-fb01-4dd1-abcb-61a0a6dc4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:10.074029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814127a-fb01-4dd1-abcb-61a0a6dc4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.074336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814127a-fb01-4dd1-abcb-61a0a6dc4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.074607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814127a-fb01-4dd1-abcb-61a0a6dc4cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.106562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:10.138605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52f7f9-7e93-48fd-8259-57e687c0b16a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:10.159742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814127a-fb01-4dd1-abcb-61a0a6dc4cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:10.161131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814127a-fb01-4dd1-abcb-61a0a6dc4cb0 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:10.181776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:10.183262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.183262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.183262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.186247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:10.187681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:10.189947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 174/348] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:09 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:10.191647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:10.259877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:10.287409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 175/349] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:10.290342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91617b0-0840-45bd-a8d5-72c8e6850557 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.290342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91617b0-0840-45bd-a8d5-72c8e6850557 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.290342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91617b0-0840-45bd-a8d5-72c8e6850557 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.292828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 176/350] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:10.294359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959bf9d-0167-47f5-bdfd-6c7b8f2168d0 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.294653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959bf9d-0167-47f5-bdfd-6c7b8f2168d0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.294930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959bf9d-0167-47f5-bdfd-6c7b8f2168d0 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.296296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 177/351] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:10.296997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959bf9d-0167-47f5-bdfd-6c7b8f2168d0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:10.317327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938fff8-5b3d-4ca7-9d94-15a2f706a704 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:10.358291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457118-5be2-43b0-a2dc-c2d513a76e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:10.358865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457118-5be2-43b0-a2dc-c2d513a76e67 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.359150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457118-5be2-43b0-a2dc-c2d513a76e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.359413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457118-5be2-43b0-a2dc-c2d513a76e67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.361013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:10.417975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959bf9d-0167-47f5-bdfd-6c7b8f2168d0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:10.439828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457118-5be2-43b0-a2dc-c2d513a76e67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:10.441759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457118-5be2-43b0-a2dc-c2d513a76e67 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:10.465127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:10.465772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.466533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.466816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.474530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:10.475987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:10.477628 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 178/352] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:10 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:10.479385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:10.508148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:10.566408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 175/353] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:10.568105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93fe1-f8d6-4530-a846-f86c800988d3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.568393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93fe1-f8d6-4530-a846-f86c800988d3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.568668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93fe1-f8d6-4530-a846-f86c800988d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.589808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-83b93fe1-f8d6-4530-a846-f86c800988d3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:10.591133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-83b93fe1-f8d6-4530-a846-f86c800988d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:10.607772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f822352-67f4-4648-9bdd-21c3f31db145 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:10.623395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:10.648169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce46b7-72eb-4b50-8185-0764c40e2f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:10.648815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce46b7-72eb-4b50-8185-0764c40e2f14 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.649091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce46b7-72eb-4b50-8185-0764c40e2f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.649356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce46b7-72eb-4b50-8185-0764c40e2f14 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.650563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce46b7-72eb-4b50-8185-0764c40e2f14 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:10.652009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce46b7-72eb-4b50-8185-0764c40e2f14 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:10.653665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 179/354] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:10 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:10.654796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5b3b-b1e3-4cb5-b8a7-d1cdaee12d5d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.655257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5b3b-b1e3-4cb5-b8a7-d1cdaee12d5d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.655529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5b3b-b1e3-4cb5-b8a7-d1cdaee12d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.674317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 176/355] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:10.675562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56348-69e6-48af-b610-9ace489767c0 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.675669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-188a5b3b-b1e3-4cb5-b8a7-d1cdaee12d5d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:10.675854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56348-69e6-48af-b610-9ace489767c0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.676116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56348-69e6-48af-b610-9ace489767c0 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.677212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-188a5b3b-b1e3-4cb5-b8a7-d1cdaee12d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:10.678686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 177/356] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:10.679636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56348-69e6-48af-b610-9ace489767c0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:10.762418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 180/357] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:10.763579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5b3b-b1e3-4cb5-b8a7-d1cdaee12d5d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:10.787619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:10.805805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56348-69e6-48af-b610-9ace489767c0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:10.851259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:10.851883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.852163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.852436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.853229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=1545cd354e894182b36f434125235edd) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:10.853690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:10.856062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:10.862581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 178/358] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:10 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=1545cd354e894182b36f434125235edd&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:10.863362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:10.889418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5b3b-b1e3-4cb5-b8a7-d1cdaee12d5d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:10.905543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:10.941235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13fc8ab-38be-43be-942b-81a3e33708be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:10.942320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fc8ab-38be-43be-942b-81a3e33708be None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.943005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fc8ab-38be-43be-942b-81a3e33708be None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.943501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13fc8ab-38be-43be-942b-81a3e33708be None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.945663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fc8ab-38be-43be-942b-81a3e33708be None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:10.947436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13fc8ab-38be-43be-942b-81a3e33708be None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:10.949380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 181/359] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:10 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:10.950443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebe65-9eb4-4de6-9c2b-891f2dd0c884 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:10.950869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebe65-9eb4-4de6-9c2b-891f2dd0c884 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:10.951269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebe65-9eb4-4de6-9c2b-891f2dd0c884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:10.976575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-520ebe65-9eb4-4de6-9c2b-891f2dd0c884 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:10.978058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-520ebe65-9eb4-4de6-9c2b-891f2dd0c884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:11.017745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b966e1-7a6a-45fa-b53c-a60e60c15a73 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.063117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 182/360] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:11.068154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ebe65-9eb4-4de6-9c2b-891f2dd0c884 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.070769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.071692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.072117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.072534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.073863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:11.075668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:11.077416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 179/361] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:10 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:11.080377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.112683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:11.205838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ebe65-9eb4-4de6-9c2b-891f2dd0c884 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.218637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b16a-a5fe-4c75-bc7e-e8e9a9ffd781 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.222337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:11.254010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.254702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.255187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.255289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.264520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.264898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.265180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.265457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.269542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:11.275816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:11.276697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 27 08:20:11.277487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:11.280856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 183/362] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:11.280856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.283153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:11.288493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:11.289388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:11.290256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:11.291796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 180/363] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:11.292614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.386530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:11.402998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c1a45-6ab4-4b10-ba35-973f97459951 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.423093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b138f-74d7-41c6-addf-36cb2137c239 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.455169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.456861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.457359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.457863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.459315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:11.460972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:11.462986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 184/364] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:11.463758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.476119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.477126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.477412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.477686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.493580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:11.498552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:11.499400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 181/365] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:11.500604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.524933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:11.619972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f55ba7-d4d9-43bc-8bc2-0ad41610afe9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.652784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f00ef-8451-4bf3-ab76-46d82f42ee8c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.671134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.671918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.672203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.672766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.673568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=1545cd354e894182b36f434125235edd) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:11.673882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:11.676186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:11.685798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 185/366] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=1545cd354e894182b36f434125235edd&role.id=d469c6f0fa524f36949e61ae51807884 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:11.686495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.693927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.694590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.694912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.695180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.695918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:11.696221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:11.697626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:11.703792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:11.708400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 182/367] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:11.715327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.850578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07faa7-2ea6-4422-93d9-280559ddb1d7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.853895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192cdb8-0c1e-41c4-af2f-e592414b3fb6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:11.855730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:11.900321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.900321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.900321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.900669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.902379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=9e85f13607084c63b3cb16e16079843e) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:11.902379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:11.903005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:11.910075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:11.910727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:11.911009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:11.911283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:11.912007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:11.912314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:11.913711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:11.918304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 183/368] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:11.919327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:11.924732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 186/369] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:11.925660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:12.012883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:12.065461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c72423-c034-46ab-9f53-f56623c1ce60 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:12.092887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4504c0d-f315-4251-9717-b37ab90360cb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:12.118848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:12.119513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:12.119795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:12.120064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] PATH_INFO: `/v3/users/0c2e67a074e84e1ba5003585aff31e0e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:12.121288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] RBAC: Authorizing `identity:get_user(user_id=0c2e67a074e84e1ba5003585aff31e0e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:12.122784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:12.129356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 187/370] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/users/0c2e67a074e84e1ba5003585aff31e0e => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:12.131405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:12.164700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:12.165324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:12.165603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:12.165878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:12.166619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:12.166921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:12.169441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:12.183579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 184/371] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:12.184481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:12.202675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:12.312359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d48b7-3b15-4257-bbdc-30a9028fb45f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:12.322927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f6964-e700-4fa6-8eb8-dcb6e37dac3a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:12.351837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:12.368131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:12.373812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:12.373812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:12.373812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:12.373812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] RBAC: Adding query filter params (role.id=ffc01dcb00834115860b2bf1f7390419, user.id=0c2e67a074e84e1ba5003585aff31e0e, scope.system=all) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:12.373812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:12.373812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:12.379585 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:12.380208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:12.380488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:12.380759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:12.404102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 185/372] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 27 08:20:12 2022] GET /identity/v3/role_assignments?user.id=0c2e67a074e84e1ba5003585aff31e0e&scope.system=all&role.id=ffc01dcb00834115860b2bf1f7390419 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:12.404979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:12.497868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:12.500572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:12.542616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:12.633870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d7cda-31ff-4422-b49e-85da7667b917 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:12.677457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 188/373] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:12.681632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:12.716508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:12.717166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:12.717407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:12.717678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:12.826169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:12.853904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:12.857271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:12.912221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c8ded-fd8c-4a9e-8e29-5b2ae18167e7 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:12.980670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 186/374] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:12.980857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:12.988102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:12.992397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:12.997188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:12.997188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:12.997188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:12.997188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:12.997188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:12.998564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 189/375] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:12 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:13.005162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:13.138085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:13.169996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea641d-fc29-48ba-808f-68b074e728c8 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:13.183211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033f36-29ba-4c0d-a833-0679c5da9644 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:13.221252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d519b5d-933c-4de2-afea-3b001700ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:13.222254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519b5d-933c-4de2-afea-3b001700ece3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.222531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519b5d-933c-4de2-afea-3b001700ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.222809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d519b5d-933c-4de2-afea-3b001700ece3 None admin] PATH_INFO: `/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.224010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d519b5d-933c-4de2-afea-3b001700ece3 None admin] RBAC: Authorizing `identity:get_project(project_id=28a37ee923b244c5bc101a0c6c5cd5d6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:13.225455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d519b5d-933c-4de2-afea-3b001700ece3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:13.225829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55109b-3f96-4b2a-a328-85e2cb7079e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:13.226607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55109b-3f96-4b2a-a328-85e2cb7079e9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.226980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55109b-3f96-4b2a-a328-85e2cb7079e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.227145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 187/376] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:12 2022] GET /identity/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6 => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:13.228377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55830-654b-4b53-a900-d91c01f2b92e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.228661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55830-654b-4b53-a900-d91c01f2b92e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.228935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55830-654b-4b53-a900-d91c01f2b92e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.230132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 188/377] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:13.230371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55109b-3f96-4b2a-a328-85e2cb7079e9 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.231608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55109b-3f96-4b2a-a328-85e2cb7079e9 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:13.233051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55109b-3f96-4b2a-a328-85e2cb7079e9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:13.235061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55830-654b-4b53-a900-d91c01f2b92e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:13.235948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 190/378] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:13 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:13.243568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef781c-6d4e-4e92-be32-eae6f1fb1d42 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.243848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef781c-6d4e-4e92-be32-eae6f1fb1d42 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.244127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef781c-6d4e-4e92-be32-eae6f1fb1d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.267993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-baef781c-6d4e-4e92-be32-eae6f1fb1d42 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:13.270464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-baef781c-6d4e-4e92-be32-eae6f1fb1d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:13.356854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:13.364580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 191/379] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:13.365424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef781c-6d4e-4e92-be32-eae6f1fb1d42 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:13.382958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55830-654b-4b53-a900-d91c01f2b92e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:13.426343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:13.426913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.427195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.427467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] PATH_INFO: `/v3/groups/24e97216d8ae44719a659585de8a7cb3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.435040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] RBAC: Authorizing `identity:get_group(group_id=24e97216d8ae44719a659585de8a7cb3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:13.436516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:13.438557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 189/380] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:13 2022] GET /identity/v3/groups/24e97216d8ae44719a659585de8a7cb3 => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:13.439339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:13.462562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:13.516526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef781c-6d4e-4e92-be32-eae6f1fb1d42 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:13.561442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effe1ea-5f44-4ae5-a08f-fa54060a7328 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:13.566698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:13.567326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.567607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.567878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.568668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=9e85f13607084c63b3cb16e16079843e) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:13.568974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:13.577712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:13.578223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 192/381] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:13 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=d469c6f0fa524f36949e61ae51807884 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:13.579084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:13.610806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:13.611700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.612811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.613290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.625201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:13.631432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:13.632491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:13.633284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:13.636091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 190/382] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:13.651235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:13.664978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:13.716351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79cbf5-79e8-4a36-a214-4b60ad537f02 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:13.757422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:13.758547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.759040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.759463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.760461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] RBAC: Adding query filter params (group.id=24e97216d8ae44719a659585de8a7cb3, role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=28a37ee923b244c5bc101a0c6c5cd5d6) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:13.760922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:13.762760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:13.764151 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:13.772064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 193/383] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:13 2022] GET /identity/v3/role_assignments?group.id=24e97216d8ae44719a659585de8a7cb3&scope.project.id=28a37ee923b244c5bc101a0c6c5cd5d6&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:13.797397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54961a11-6d74-47b6-8491-0647012230d6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:13.840594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:13.841580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:13.842058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:13.842547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:13.843469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:13.843974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:13.846012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:13.856685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 191/384] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:13.862797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:13.977529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63938b-9512-4808-bbd7-1e258e2c970a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:14.004784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:14.014539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:14.019694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.019694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.019694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.019694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:14.019694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:14.021677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 194/385] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:13 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:14.026919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:14.136839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ce6db-a08d-46ae-9934-40441ab16125 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:14.172731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f103ce-197d-42ba-9770-0ded33a439f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:14.173690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f103ce-197d-42ba-9770-0ded33a439f2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.174182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f103ce-197d-42ba-9770-0ded33a439f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.175181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f103ce-197d-42ba-9770-0ded33a439f2 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.176530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f103ce-197d-42ba-9770-0ded33a439f2 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:14.178175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f103ce-197d-42ba-9770-0ded33a439f2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:14.180444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 192/386] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:14 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:14.190984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:14.266101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803672-43b6-48ad-90af-5e56fda5c804 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.266688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803672-43b6-48ad-90af-5e56fda5c804 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.267148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803672-43b6-48ad-90af-5e56fda5c804 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.268711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 193/387] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:20:14.274674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef1151-2ea4-400e-81e7-16bb2fc44163 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.274674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef1151-2ea4-400e-81e7-16bb2fc44163 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.274674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef1151-2ea4-400e-81e7-16bb2fc44163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.293730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:14.311504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3ff8-c234-4c24-9a39-56264e32ef78 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:14.327945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b4ef1151-2ea4-400e-81e7-16bb2fc44163 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:14.327945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b4ef1151-2ea4-400e-81e7-16bb2fc44163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:14.350983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2c1e2-feaf-4960-994d-5bcd661e544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:14.351912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c1e2-feaf-4960-994d-5bcd661e544e None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.352338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c1e2-feaf-4960-994d-5bcd661e544e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.353698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c1e2-feaf-4960-994d-5bcd661e544e None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.356551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2c1e2-feaf-4960-994d-5bcd661e544e None admin] RBAC: Authorizing `identity:create_grant(project_id=1545cd354e894182b36f434125235edd, group_id=6c8372ca2d4f481c8097cc7b0377c799, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:14.358189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2c1e2-feaf-4960-994d-5bcd661e544e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:14.423107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 194/388] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:14.445480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef1151-2ea4-400e-81e7-16bb2fc44163 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:14.485399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 195/389] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 27 08:20:14 2022] PUT /identity/v3/projects/1545cd354e894182b36f434125235edd/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:14.491239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa4e48-f63b-4477-ac37-c5ff959c74de None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.491530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa4e48-f63b-4477-ac37-c5ff959c74de None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.506326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa4e48-f63b-4477-ac37-c5ff959c74de None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.506326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 196/390] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:14.506326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011399cc-0dd2-4c3c-90a7-c874b219d98a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.506326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011399cc-0dd2-4c3c-90a7-c874b219d98a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.506326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011399cc-0dd2-4c3c-90a7-c874b219d98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.518993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-011399cc-0dd2-4c3c-90a7-c874b219d98a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:14.520207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-011399cc-0dd2-4c3c-90a7-c874b219d98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:14.585825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef1151-2ea4-400e-81e7-16bb2fc44163 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:14.587759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:14.629677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 197/391] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:14.640333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:14.641001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.641286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.641559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.644414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:14.655343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011399cc-0dd2-4c3c-90a7-c874b219d98a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:14.661854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:14.666898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:14.670931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:14.671795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 195/392] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:20:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:14.675183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:14.788414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab130d-d318-491f-81a1-935675f1f9eb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:14.828061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:14.828969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.829394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.829823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.831422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:14.831992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:14.832449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011399cc-0dd2-4c3c-90a7-c874b219d98a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:14.834127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:14.849250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 196/393] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:14.854742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:14.878027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:14.879230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:14.880412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:14.880988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:14.901835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:14.911046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:14.912485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:14.913397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:14.916978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 198/394] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:14.922997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:14.963912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:15.022587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:15.039522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b235247-2cbf-4b66-ac93-ae839d5dcef3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:15.105439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:15.106377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:15.106817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:15.107245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:15.109876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee55f-df8e-4038-9687-e3817152da05 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:15.110680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:15.114540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:15.117378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 197/395] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:14 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:15.123160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:15.162733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:15.163633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:15.164061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:15.164486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:15.165354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:15.165895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:15.167534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:15.178188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 199/396] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:15.187740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:15.296891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03052a-57c3-4f65-9f4d-f96ca79f2cae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:15.323500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:15.343963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:15.344822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:15.345253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:15.345993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:15.347989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:15.349614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:15.351423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 198/397] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:15 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:15.357133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:15.381054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8c43-fdff-4dda-9dd4-b818fd953c71 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:15.390474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:15.441151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:15.441902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:15.442195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:15.442472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:15.446431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:15.447985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:15.450021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 200/398] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:15 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:15.455201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:15.519208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea840242-ac66-43e8-9117-e5b422314c1f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:15.575390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc50e7a-f6ee-477e-92eb-bcc14ac8e65e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:15.582972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb103c-ad31-492e-9680-7b73561b3351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:15.583664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb103c-ad31-492e-9680-7b73561b3351 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:15.583940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb103c-ad31-492e-9680-7b73561b3351 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:15.584209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb103c-ad31-492e-9680-7b73561b3351 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:15.586656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb103c-ad31-492e-9680-7b73561b3351 None admin] RBAC: Authorizing `identity:create_grant(project_id=1545cd354e894182b36f434125235edd, group_id=6c8372ca2d4f481c8097cc7b0377c799, role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:15.588100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb103c-ad31-492e-9680-7b73561b3351 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:15.612180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:15.612866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:15.614039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:15.614039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:15.614992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:15.616656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:15.618850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 201/399] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:15 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:15.626637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:15.673023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:15.719804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 199/400] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 27 08:20:15 2022] PUT /identity/v3/projects/1545cd354e894182b36f434125235edd/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/d469c6f0fa524f36949e61ae51807884 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:20:15.745335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e695d-9906-4b76-ae2e-06d8066bd232 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:15.800829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef68c8-87b8-439a-98ea-116a9b0e55a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:15.801901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef68c8-87b8-439a-98ea-116a9b0e55a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:15.802435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef68c8-87b8-439a-98ea-116a9b0e55a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:15.802909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef68c8-87b8-439a-98ea-116a9b0e55a7 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:15.808161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef68c8-87b8-439a-98ea-116a9b0e55a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e85f13607084c63b3cb16e16079843e, group_id=6c8372ca2d4f481c8097cc7b0377c799, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:15.810068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef68c8-87b8-439a-98ea-116a9b0e55a7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:15.931922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 202/401] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 27 08:20:15 2022] PUT /identity/v3/projects/9e85f13607084c63b3cb16e16079843e/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:16.188287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23ee08-04eb-475b-ad5c-6a8d50e31a59 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.188780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23ee08-04eb-475b-ad5c-6a8d50e31a59 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.188780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23ee08-04eb-475b-ad5c-6a8d50e31a59 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.190530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 200/402] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:16.196450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95640450-c8fc-4006-94ac-f2ec5250d0dd None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.196760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95640450-c8fc-4006-94ac-f2ec5250d0dd None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.197004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95640450-c8fc-4006-94ac-f2ec5250d0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.224000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-95640450-c8fc-4006-94ac-f2ec5250d0dd None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:16.225181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-95640450-c8fc-4006-94ac-f2ec5250d0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:16.292438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476e462-1c4f-4a92-9b9a-f970bfc4fe54 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.292779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476e462-1c4f-4a92-9b9a-f970bfc4fe54 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.293001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476e462-1c4f-4a92-9b9a-f970bfc4fe54 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.294662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 201/403] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:16.300102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11a5a-d261-4f02-983e-19e062b18da4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.300740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11a5a-d261-4f02-983e-19e062b18da4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.301226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a11a5a-d261-4f02-983e-19e062b18da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.319581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-82a11a5a-d261-4f02-983e-19e062b18da4 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:16.321098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-82a11a5a-d261-4f02-983e-19e062b18da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:16.332273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 203/404] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:16.374029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95640450-c8fc-4006-94ac-f2ec5250d0dd None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:16.391582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:16.426475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 202/405] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:16.442940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:16.478294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a11a5a-d261-4f02-983e-19e062b18da4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:16.572211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95640450-c8fc-4006-94ac-f2ec5250d0dd None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:16.629122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:16.630164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.631980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.632456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.651661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:16.657896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:16.658913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:16.659697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:16.670352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 204/406] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:16.673092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:16.677529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a11a5a-d261-4f02-983e-19e062b18da4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:16.751135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:16.752132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.752643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.753109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.775988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:16.782703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:16.784214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:16.785022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:16.795916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 203/407] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:20:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:16.806710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:16.857808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:16.873987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e8cf4-4520-4cc3-b9e6-63ad3aac51ee None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:16.898481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:16.936563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da96fa-864c-4a06-8a41-e3e2572549f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:16.937331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da96fa-864c-4a06-8a41-e3e2572549f3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.937511 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da96fa-864c-4a06-8a41-e3e2572549f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.938617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da96fa-864c-4a06-8a41-e3e2572549f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.939421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da96fa-864c-4a06-8a41-e3e2572549f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:16.940438 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da96fa-864c-4a06-8a41-e3e2572549f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:16.942351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da96fa-864c-4a06-8a41-e3e2572549f3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:16.953275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 205/408] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:16.954696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dfdd7-615f-4de2-bccd-4ca148f66bb8 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:16.955059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dfdd7-615f-4de2-bccd-4ca148f66bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:16.955460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dfdd7-615f-4de2-bccd-4ca148f66bb8 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:16.960926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 206/409] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:16.961785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dfdd7-615f-4de2-bccd-4ca148f66bb8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:16.976698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a39b8-8a92-4f58-b33a-378248eedda2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.015470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e986b5-75b8-47b1-81f5-b1ce039b7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.016143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e986b5-75b8-47b1-81f5-b1ce039b7563 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.016475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e986b5-75b8-47b1-81f5-b1ce039b7563 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.016828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e986b5-75b8-47b1-81f5-b1ce039b7563 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.017605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e986b5-75b8-47b1-81f5-b1ce039b7563 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:17.018062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e986b5-75b8-47b1-81f5-b1ce039b7563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.019557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e986b5-75b8-47b1-81f5-b1ce039b7563 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.028474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 204/410] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:17.029858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fdb65-190e-471d-ae6e-a1b8e651da3e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.032115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fdb65-190e-471d-ae6e-a1b8e651da3e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.032465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fdb65-190e-471d-ae6e-a1b8e651da3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.052547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5d4fdb65-190e-471d-ae6e-a1b8e651da3e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:17.053945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d4fdb65-190e-471d-ae6e-a1b8e651da3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:17.099446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dfdd7-615f-4de2-bccd-4ca148f66bb8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.135143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 205/411] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:17.136282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fdb65-190e-471d-ae6e-a1b8e651da3e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:17.144339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.145063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.145418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.146180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] PATH_INFO: `/v3/groups/24e97216d8ae44719a659585de8a7cb3` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.148912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] RBAC: Authorizing `identity:get_group(group_id=24e97216d8ae44719a659585de8a7cb3)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.150664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.152447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 207/412] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:16 2022] GET /identity/v3/groups/24e97216d8ae44719a659585de8a7cb3 => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:17.157868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:17.177171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:17.195912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:17.271082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fdb65-190e-471d-ae6e-a1b8e651da3e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.288837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98701a-03a0-4129-a5a0-ee56224b7aa6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.319065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.320102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.320634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.320913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.322337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.324401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.326345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 206/413] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:17 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:17.327183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:17.327839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.328665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.329099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.329619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] PATH_INFO: `/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.331161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] RBAC: Authorizing `identity:get_project(project_id=28a37ee923b244c5bc101a0c6c5cd5d6)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.332828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.334980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 208/414] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:17 2022] GET /identity/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:17.335996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:17.440347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:17.451529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeacbda-f02c-45b6-862b-a64de87db7ae None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.461187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fbf7a-6338-47f3-af72-a93b51da4e7b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.463101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:17.492215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.493267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.494080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.496589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.496589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.498547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.500666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 209/415] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:17 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:17.501607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:17.506956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.507791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.510521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.510899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.525444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:17.531574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:17.532686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:17.534374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:17.540474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 207/416] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:17.540474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:17.645855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf4868-c5e6-41e7-8690-507f68710065 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.661274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e553d8-948a-4455-92d9-1f2b760ebdbb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.693472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.694416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.694838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.695251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] PATH_INFO: `/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6/groups/24e97216d8ae44719a659585de8a7cb3/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.696417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.697194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.697565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.697915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] RBAC: Authorizing `identity:create_grant(project_id=28a37ee923b244c5bc101a0c6c5cd5d6, group_id=24e97216d8ae44719a659585de8a7cb3, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.698026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.699673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.700388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] RBAC: Authorizing `identity:create_grant(project_id=9e85f13607084c63b3cb16e16079843e, group_id=6c8372ca2d4f481c8097cc7b0377c799, role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.701891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.767023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 210/417] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 27 08:20:17 2022] PUT /identity/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6/groups/24e97216d8ae44719a659585de8a7cb3/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:17.768913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:17.795020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:17.802247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 208/418] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 27 08:20:17 2022] PUT /identity/v3/projects/9e85f13607084c63b3cb16e16079843e/groups/6c8372ca2d4f481c8097cc7b0377c799/roles/d469c6f0fa524f36949e61ae51807884 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:17.904076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f126004-4564-40b6-9a90-561c61fe8a39 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:17.954241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065b6ec-1581-4a67-a2dd-81cf6d0d8b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:17.956493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065b6ec-1581-4a67-a2dd-81cf6d0d8b73 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:17.957159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065b6ec-1581-4a67-a2dd-81cf6d0d8b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:17.957626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065b6ec-1581-4a67-a2dd-81cf6d0d8b73 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:17.958537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065b6ec-1581-4a67-a2dd-81cf6d0d8b73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:17.958986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065b6ec-1581-4a67-a2dd-81cf6d0d8b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:17.960558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065b6ec-1581-4a67-a2dd-81cf6d0d8b73 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:17.974291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 211/419] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:17.980258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:18.010948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c0b43-793f-4c25-8909-3761e29d4761 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.013479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c0b43-793f-4c25-8909-3761e29d4761 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.013479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c0b43-793f-4c25-8909-3761e29d4761 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.014436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:18.016145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 212/420] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:20:18.028826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1b206-54ce-4521-a944-8cc6941b6659 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.029341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1b206-54ce-4521-a944-8cc6941b6659 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.029763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1b206-54ce-4521-a944-8cc6941b6659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.052710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-92a1b206-54ce-4521-a944-8cc6941b6659 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:18.053865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-92a1b206-54ce-4521-a944-8cc6941b6659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:18.127866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46192af8-1d05-4e94-9c01-0dbd63d5ca3a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:18.148877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 213/421] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:20:18.168128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:18.168807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.169104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.169371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.171364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1b206-54ce-4521-a944-8cc6941b6659 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:18.186756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:18.298299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:18.301506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:18.327595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:18.335740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1b206-54ce-4521-a944-8cc6941b6659 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:18.384961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072455a9-63d3-4be3-8110-4cb4e3d690a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:18.385625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072455a9-63d3-4be3-8110-4cb4e3d690a7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.385908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072455a9-63d3-4be3-8110-4cb4e3d690a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.386206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072455a9-63d3-4be3-8110-4cb4e3d690a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.407141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072455a9-63d3-4be3-8110-4cb4e3d690a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-072455a9-63d3-4be3-8110-4cb4e3d690a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:18.413213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:18.414335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:18.415407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:18.417157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 214/422] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:20:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:18.421878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d530e2-a573-45ff-b385-6c5f7fc8129b None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.421878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d530e2-a573-45ff-b385-6c5f7fc8129b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.421878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d530e2-a573-45ff-b385-6c5f7fc8129b None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.421878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 215/423] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:18.422699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 209/424] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:18.423045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d530e2-a573-45ff-b385-6c5f7fc8129b None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:18.427880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:18.504672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:18.546020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d530e2-a573-45ff-b385-6c5f7fc8129b None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:18.559987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407491b7-6cfc-4d49-a8c5-c5a6a9a3c3be None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:18.588930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:18.594240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:18.595029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.595467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.596204 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.597196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:18.597658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:18.599285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:18.601291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33811a-28a9-4082-b1e7-3b9766f5d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:18.602882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33811a-28a9-4082-b1e7-3b9766f5d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.602882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33811a-28a9-4082-b1e7-3b9766f5d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.602882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33811a-28a9-4082-b1e7-3b9766f5d1d0 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.603974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33811a-28a9-4082-b1e7-3b9766f5d1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:18.605780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33811a-28a9-4082-b1e7-3b9766f5d1d0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:18.607556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 210/425] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:18 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:18.608810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec2bc2-0b37-4c93-a3c2-e6a82af0aecb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.609136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec2bc2-0b37-4c93-a3c2-e6a82af0aecb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.609423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec2bc2-0b37-4c93-a3c2-e6a82af0aecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.609657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 216/426] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:18.615739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:18.636815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3dec2bc2-0b37-4c93-a3c2-e6a82af0aecb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:18.638330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3dec2bc2-0b37-4c93-a3c2-e6a82af0aecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:18.724321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 211/427] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:18.725252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec2bc2-0b37-4c93-a3c2-e6a82af0aecb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:18.770560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2f4b8-4338-42c9-9536-ac1fbd3da65f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:18.816228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:18.826260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:18.830844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.830844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.830844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.830844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:18.831458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:18.834486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 217/428] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:18 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:18.835573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:18.859951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec2bc2-0b37-4c93-a3c2-e6a82af0aecb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:18.886246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:18.904935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:18.905951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.906459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.906932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.950063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2964-9e68-49a8-ac27-c7c86eea6bab None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:18.985054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:18.985713 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:18.985948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:18.986260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:18.987439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:18.989376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:18.996773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:19.002901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:19.004085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:19.004879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 218/429] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:19.005524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:19.072526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:19.126487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa44bb1-2014-4056-8f0f-e1388a49926e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:19.135879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 212/430] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:19.136913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:19.162617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:19.167377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:19.168301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:19.168732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:19.169144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:19.170100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=1545cd354e894182b36f434125235edd) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:19.170591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:19.174900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:19.179836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 219/431] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=1545cd354e894182b36f434125235edd&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:19.180715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:19.256065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd1814-176d-41a6-8f9f-1c9d0b1ccc24 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:19.293409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8568605-cb9a-4f37-b842-4b2b7fe14b23 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:19.306603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:19.307427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:19.307811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:19.308193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:19.309267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:19.309687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:19.311232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:19.321722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 213/432] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:19.327516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:19.362612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:19.386379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:19.386379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:19.386379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:19.386379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] PATH_INFO: `/v3/projects/1545cd354e894182b36f434125235edd` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:19.386777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] RBAC: Authorizing `identity:get_project(project_id=1545cd354e894182b36f434125235edd)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:19.386777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:19.388652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 220/433] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/projects/1545cd354e894182b36f434125235edd => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:19.404020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:19.438810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:19.452704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae55342-51c0-4da2-8fed-90f7b93a23b5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:19.491778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:19.492504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:19.492900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:19.493283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:19.522900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933538a-a131-4dbe-a442-ae3e60d2b803 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:19.558085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:19.558765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:19.559042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:19.559312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:19.560467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:19.561907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:19.563794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 221/434] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:19.570777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:19.583110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:19.584856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:19.635297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:19.680057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762a4ac-06cd-47d7-8db7-3804ecf1a286 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:19.686431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:19.700228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 214/435] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:19.706210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:19.719532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:19.720154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:19.720507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:19.721691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:19.722558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=1545cd354e894182b36f434125235edd) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:19.722870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:19.724316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:19.731379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 222/436] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=1545cd354e894182b36f434125235edd&role.id=d469c6f0fa524f36949e61ae51807884 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:19.824627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44b647-9c71-4225-a513-3fe24732cbc6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:19.865361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:19.866023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:19.866339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:19.866616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:19.867871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:19.869353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:19.871452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 215/437] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:19.877299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.010297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe3cca4-76dc-4134-96df-ae4df4a620df None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.036738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.050731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78150fc-9f09-4bdb-91d7-c9b54f7c1c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.051897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78150fc-9f09-4bdb-91d7-c9b54f7c1c36 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.052377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78150fc-9f09-4bdb-91d7-c9b54f7c1c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.052856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78150fc-9f09-4bdb-91d7-c9b54f7c1c36 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.054243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78150fc-9f09-4bdb-91d7-c9b54f7c1c36 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:20.056043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78150fc-9f09-4bdb-91d7-c9b54f7c1c36 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:20.058439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 223/438] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:19 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:20.063896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.122115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d624bc-0386-40bf-902e-935392d0a533 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.122454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d624bc-0386-40bf-902e-935392d0a533 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.122699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d624bc-0386-40bf-902e-935392d0a533 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.124047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 224/439] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:20:20.125676 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.130634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b90e59-5024-4565-90c1-a0513259ddd2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.130912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b90e59-5024-4565-90c1-a0513259ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.131191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b90e59-5024-4565-90c1-a0513259ddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.149864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e7b90e59-5024-4565-90c1-a0513259ddd2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:20.151119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e7b90e59-5024-4565-90c1-a0513259ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:20.182814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f3043-f108-4ecc-a68d-4f997aa27e8c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.228815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ceb9bf-b411-4d0d-911a-7cab53f82e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.229591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ceb9bf-b411-4d0d-911a-7cab53f82e3f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.229873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ceb9bf-b411-4d0d-911a-7cab53f82e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.230500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ceb9bf-b411-4d0d-911a-7cab53f82e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.231305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ceb9bf-b411-4d0d-911a-7cab53f82e3f None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=6a60961045ef46cd91a802e9a6cf7b9e, scope.project.id=9e85f13607084c63b3cb16e16079843e) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:20.231896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ceb9bf-b411-4d0d-911a-7cab53f82e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:20.233336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ceb9bf-b411-4d0d-911a-7cab53f82e3f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:20.236191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 225/440] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:20.241044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 216/441] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=6a60961045ef46cd91a802e9a6cf7b9e => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:20.253012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8382bf-1634-47d4-8db2-1a62ab5280fe None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.253524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8382bf-1634-47d4-8db2-1a62ab5280fe None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.253822 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8382bf-1634-47d4-8db2-1a62ab5280fe None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.255690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 226/442] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:20.261577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48555e-6751-401c-8a57-4cfb0124f807 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.261752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48555e-6751-401c-8a57-4cfb0124f807 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.262021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48555e-6751-401c-8a57-4cfb0124f807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.262384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8382bf-1634-47d4-8db2-1a62ab5280fe None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.281947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ec48555e-6751-401c-8a57-4cfb0124f807 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:20.283214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ec48555e-6751-401c-8a57-4cfb0124f807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:20.379634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 217/443] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:20.403137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48555e-6751-401c-8a57-4cfb0124f807 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.406054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8382bf-1634-47d4-8db2-1a62ab5280fe None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.431918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.444024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.450040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.450040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.450040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.456525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:20.461012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:20.461557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:20.462633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:20.463429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 227/444] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:20.469090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.527455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48555e-6751-401c-8a57-4cfb0124f807 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.580647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.581496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.581790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.582063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.593226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:20.593368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3402d0f-4a8e-4971-9f94-0c05b4211020 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:20.600082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:20.601165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:20.602244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 218/445] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:20.607092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.635533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.636150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.636423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.636694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.637409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:20.637708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:20.639115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:20.647742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 228/446] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:20.653347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.697431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.712855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671af70-84c3-4c99-a5e2-f9aa3dc008f7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.715125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.746167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.746765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.747047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.747313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.748101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:20.748406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:20.749772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:20.750948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acd0c8-faf2-4071-b435-152fc2d2c552 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.758547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 219/447] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:20.763956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.786309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.786869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.787146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.787412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.852970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:20.854436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:20.863166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9eea4-569a-4ecf-bdee-237e1f3c0a61 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:20.896376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.896545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:20.897011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:20.897291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:20.897571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:20.930946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:20.950906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 229/448] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 08:20:20.957228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:20.966153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:20.967625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:21.057092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde3fc8-1afb-4297-8c8a-4001ef8f1ab5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:21.068583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 220/449] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 08:20:21.074247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:21.090980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:21.091615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:21.091893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:21.092164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] PATH_INFO: `/v3/projects/9e85f13607084c63b3cb16e16079843e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:21.093313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] RBAC: Authorizing `identity:get_project(project_id=9e85f13607084c63b3cb16e16079843e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:21.094785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:21.097499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 230/450] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:20 2022] GET /identity/v3/projects/9e85f13607084c63b3cb16e16079843e => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:21.102793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:21.117571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:21.136997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:21.170097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424eac0-e592-4d6d-b726-e95b74a8b054 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:21.200392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70940397-f8c4-4088-b520-9a85e2195231 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:21.203718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:21.204347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:21.204629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:21.204905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] PATH_INFO: `/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:21.206337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] RBAC: Authorizing `identity:get_project(project_id=28a37ee923b244c5bc101a0c6c5cd5d6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:21.207794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:21.209447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 221/451] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:20:21 2022] GET /identity/v3/projects/28a37ee923b244c5bc101a0c6c5cd5d6 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:21.214901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:21.234166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:21.234807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:21.235082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:21.235350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] PATH_INFO: `/v3/groups/6c8372ca2d4f481c8097cc7b0377c799` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:21.237174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] RBAC: Authorizing `identity:get_group(group_id=6c8372ca2d4f481c8097cc7b0377c799)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:21.238667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:21.240228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 231/452] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:21 2022] GET /identity/v3/groups/6c8372ca2d4f481c8097cc7b0377c799 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:21.245759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:21.312337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6253d-b68f-4cf4-9c07-8a3d265abbf0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:21.325900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:21.345930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2b932-ed97-4320-881d-3779d5e69361 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:21.346759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:21.347332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:21.347614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:21.347887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] PATH_INFO: `/v3/groups/24e97216d8ae44719a659585de8a7cb3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:21.349046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] RBAC: Authorizing `identity:get_group(group_id=24e97216d8ae44719a659585de8a7cb3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:21.350496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:21.352097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 222/453] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 27 08:20:21 2022] GET /identity/v3/groups/24e97216d8ae44719a659585de8a7cb3 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:21.357756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:21.360339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:21.381397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcfea5-fdf0-486c-b8ac-fc2c9fb9220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:21.382005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfea5-fdf0-486c-b8ac-fc2c9fb9220a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:21.382311 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfea5-fdf0-486c-b8ac-fc2c9fb9220a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:21.382583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcfea5-fdf0-486c-b8ac-fc2c9fb9220a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:21.383366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbcfea5-fdf0-486c-b8ac-fc2c9fb9220a None admin] RBAC: Adding query filter params (group.id=6c8372ca2d4f481c8097cc7b0377c799, role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=9e85f13607084c63b3cb16e16079843e) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:21.383674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbcfea5-fdf0-486c-b8ac-fc2c9fb9220a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:21.385157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbcfea5-fdf0-486c-b8ac-fc2c9fb9220a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:21.391061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 232/454] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:21 2022] GET /identity/v3/role_assignments?group.id=6c8372ca2d4f481c8097cc7b0377c799&scope.project.id=9e85f13607084c63b3cb16e16079843e&role.id=d469c6f0fa524f36949e61ae51807884 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:21.463045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496bd76-2451-404c-9060-ed3f147d137d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:21.498093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081163d5-9ab7-46f4-880e-86cec807e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:21.498752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081163d5-9ab7-46f4-880e-86cec807e489 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:21.499032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081163d5-9ab7-46f4-880e-86cec807e489 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:21.499305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081163d5-9ab7-46f4-880e-86cec807e489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:21.500104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081163d5-9ab7-46f4-880e-86cec807e489 None admin] RBAC: Adding query filter params (group.id=24e97216d8ae44719a659585de8a7cb3, role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=28a37ee923b244c5bc101a0c6c5cd5d6) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:21.500414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081163d5-9ab7-46f4-880e-86cec807e489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:21.501861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081163d5-9ab7-46f4-880e-86cec807e489 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:21.508443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 223/455] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 27 08:20:21 2022] GET /identity/v3/role_assignments?group.id=24e97216d8ae44719a659585de8a7cb3&scope.project.id=28a37ee923b244c5bc101a0c6c5cd5d6&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:23.920446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c838-35ad-4c82-8324-09079f73b546 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:23.921467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c838-35ad-4c82-8324-09079f73b546 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:23.921957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c838-35ad-4c82-8324-09079f73b546 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:23.924412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 233/456] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:23.930517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648175-f40e-48de-a2b1-f38356fd1311 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:23.931420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648175-f40e-48de-a2b1-f38356fd1311 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:23.931928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5648175-f40e-48de-a2b1-f38356fd1311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:23.954362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a5648175-f40e-48de-a2b1-f38356fd1311 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:23.956662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a5648175-f40e-48de-a2b1-f38356fd1311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:24.049235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:24.058134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 224/457] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:20:24.083791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2c838-35ad-4c82-8324-09079f73b546 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:24.197151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2c838-35ad-4c82-8324-09079f73b546 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:24.233710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90dee5-19d0-44a6-aca6-b8913d300525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:24.234764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90dee5-19d0-44a6-aca6-b8913d300525 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.235276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90dee5-19d0-44a6-aca6-b8913d300525 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.235770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90dee5-19d0-44a6-aca6-b8913d300525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.248252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90dee5-19d0-44a6-aca6-b8913d300525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:24.250009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90dee5-19d0-44a6-aca6-b8913d300525 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:24.253407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-2e90dee5-19d0-44a6-aca6-b8913d300525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:24.266977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:24.275768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:24.276625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:24.276625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:24.276625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:24.276625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:24.276625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 234/458] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:24.277140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648175-f40e-48de-a2b1-f38356fd1311 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:24.371240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709f841-08d0-490b-8aad-3b67d3d97b1d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.371786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709f841-08d0-490b-8aad-3b67d3d97b1d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.372208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709f841-08d0-490b-8aad-3b67d3d97b1d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.373946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 235/459] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:24.386028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e7d2-e86b-4a85-91d1-e3db0972dfc1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.386341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e7d2-e86b-4a85-91d1-e3db0972dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.386618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1e7d2-e86b-4a85-91d1-e3db0972dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.416829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-cfd1e7d2-e86b-4a85-91d1-e3db0972dfc1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:24.418051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cfd1e7d2-e86b-4a85-91d1-e3db0972dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:24.431674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5648175-f40e-48de-a2b1-f38356fd1311 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:24.480428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:24.480975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.481253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.481527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.482242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:24.482550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:24.483944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:24.511073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 225/460] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:24.516167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:24.556888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 236/461] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:24.558507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fa598-1385-4723-9d50-083fa6311fd6 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.564767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fa598-1385-4723-9d50-083fa6311fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.564767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fa598-1385-4723-9d50-083fa6311fd6 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.565298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 237/462] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:24.566776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd80332-66bb-4cf6-909b-625f89de9c4e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.567284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd80332-66bb-4cf6-909b-625f89de9c4e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.567772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd80332-66bb-4cf6-909b-625f89de9c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.589251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3cd80332-66bb-4cf6-909b-625f89de9c4e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:24.591271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3cd80332-66bb-4cf6-909b-625f89de9c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:24.610503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:24.660599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262be681-a791-431a-be61-5a9317dd97eb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:24.680975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 238/463] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:24.682620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd80332-66bb-4cf6-909b-625f89de9c4e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:24.696550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:24.697288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.697609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.697952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.698780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:24.700321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:24.754529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:24.797528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd80332-66bb-4cf6-909b-625f89de9c4e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:24.800629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 226/464] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:20:24 2022] POST /identity/v3/users => generated 246 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:24.801457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:24.839899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dfc87-dbfe-44d2-9af1-03f8dc19b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:24.840538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dfc87-dbfe-44d2-9af1-03f8dc19b08f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.840821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dfc87-dbfe-44d2-9af1-03f8dc19b08f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.841094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dfc87-dbfe-44d2-9af1-03f8dc19b08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.855660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dfc87-dbfe-44d2-9af1-03f8dc19b08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:24.857224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dfc87-dbfe-44d2-9af1-03f8dc19b08f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-194dfc87-dbfe-44d2-9af1-03f8dc19b08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:24.875802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:24.876756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:24.878157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:24.878157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:24.878157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:24.878157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:24.878157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 239/465] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:24.882891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f97fc-0a05-431a-9e90-e921c7964aec None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:24.883196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f97fc-0a05-431a-9e90-e921c7964aec None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:24.883464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f97fc-0a05-431a-9e90-e921c7964aec None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:24.889066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 240/466] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:24.889066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f97fc-0a05-431a-9e90-e921c7964aec None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:24.952988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6c275-1778-4c51-b3a0-49a727652582 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:24.971680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:24.999079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f541b-65f4-4b40-97a5-bcec41682e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:24.999781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f541b-65f4-4b40-97a5-bcec41682e92 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.000177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f541b-65f4-4b40-97a5-bcec41682e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.000454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f541b-65f4-4b40-97a5-bcec41682e92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.013845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f541b-65f4-4b40-97a5-bcec41682e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.015499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f541b-65f4-4b40-97a5-bcec41682e92 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.019728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f97fc-0a05-431a-9e90-e921c7964aec None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:25.030015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ce3f541b-65f4-4b40-97a5-bcec41682e92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:25.034610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:25.035450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:25.036268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:25.036268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:25.036268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:25.036268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:25.036268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 227/467] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:25.037704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5620f-c06b-450a-a7c2-45647367a3bc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.037704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5620f-c06b-450a-a7c2-45647367a3bc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.040029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5620f-c06b-450a-a7c2-45647367a3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.058393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ebb5620f-c06b-450a-a7c2-45647367a3bc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:25.063916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ebb5620f-c06b-450a-a7c2-45647367a3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:25.064068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:25.064758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.065132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.065409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.066111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:25.067472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.068960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.083308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 241/468] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:25.084247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:25.150274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 228/469] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:25.151172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5620f-c06b-450a-a7c2-45647367a3bc None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:25.197253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a3da2-b397-4211-b2a8-1aef7bedeeec None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:25.234867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:25.235961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.237102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.237588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.238718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:25.239213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.240869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.258493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 242/470] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:25.259609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:25.287211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:25.295413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5620f-c06b-450a-a7c2-45647367a3bc None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:25.298340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:25.338843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:25.339476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.339759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.340062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.341324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.342979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.390479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec81848-0e13-415e-8791-e37dd67e48ad None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:25.424275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ece5f-040c-4cc3-90ab-7f94b04d17cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:25.424898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ece5f-040c-4cc3-90ab-7f94b04d17cf None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.425174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ece5f-040c-4cc3-90ab-7f94b04d17cf None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.425548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ece5f-040c-4cc3-90ab-7f94b04d17cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.437448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ece5f-040c-4cc3-90ab-7f94b04d17cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.438969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ece5f-040c-4cc3-90ab-7f94b04d17cf None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.439986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 229/471] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:20:25 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:25.441138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-b41ece5f-040c-4cc3-90ab-7f94b04d17cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:25.461540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:25.462503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:25.463353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:25.463353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:25.463353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:25.463353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:25.463353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 243/472] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:25.470197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9bfec-319b-485b-8db8-d4597d3ec03c None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.470197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9bfec-319b-485b-8db8-d4597d3ec03c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.470197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9bfec-319b-485b-8db8-d4597d3ec03c None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.470197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 244/473] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:25.481513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9bfec-319b-485b-8db8-d4597d3ec03c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:25.570634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:25.582101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf41ebe-fd72-434d-9905-4e2f277a3eb1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:25.592838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9bfec-319b-485b-8db8-d4597d3ec03c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:25.623053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:25.623889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.624987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.624987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.625394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.627482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.634573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91378eec-e458-4195-bf8a-810a3bf2801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:25.635626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378eec-e458-4195-bf8a-810a3bf2801b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.636110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378eec-e458-4195-bf8a-810a3bf2801b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.636634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91378eec-e458-4195-bf8a-810a3bf2801b None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.637512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91378eec-e458-4195-bf8a-810a3bf2801b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:25.638020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91378eec-e458-4195-bf8a-810a3bf2801b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.639719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91378eec-e458-4195-bf8a-810a3bf2801b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.654687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 245/474] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:25.660595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82837d-2ed7-429e-8c89-8a82de7a43b1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.660890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82837d-2ed7-429e-8c89-8a82de7a43b1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.661188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82837d-2ed7-429e-8c89-8a82de7a43b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.681103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5a82837d-2ed7-429e-8c89-8a82de7a43b1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:25.682542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5a82837d-2ed7-429e-8c89-8a82de7a43b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:25.706093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 230/475] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:20:25 2022] POST /identity/v3/users => generated 248 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:25.709628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:25.722194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:25.785376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 246/476] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:25.812449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82837d-2ed7-429e-8c89-8a82de7a43b1 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:25.879350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005ce34-2b65-416b-8764-19cdc99c7c70 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:25.932869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:25.933881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:25.934537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:25.934821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:25.935098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:25.935783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:25.937226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:25.970201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82837d-2ed7-429e-8c89-8a82de7a43b1 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:26.009211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:26.009834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:26.010115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:26.010419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:26.011139 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:26.012671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:26.035039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 231/477] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:20:25 2022] POST /identity/v3/users => generated 249 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:26.056706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.sql.core [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 08:20:26.056706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 08:20:26.056706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [parameters: {'id': 'b8a7780ffb4b489695fa2a89614f6104', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 08:20:26.056706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80290) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application conn.query(q) Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result.read() Jul 27 08:20:26.075910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return ref.to_dict() Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 08:20:26.077116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application next(self.gen) Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application yield resource Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application next(self.gen) Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise self.value Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application session.commit() Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 08:20:26.078373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.session.flush() Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application compat.raise_( Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 08:20:26.079714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 08:20:26.080853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application conn.query(q) Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result.read() Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 08:20:26.081978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [parameters: {'id': 'b8a7780ffb4b489695fa2a89614f6104', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:26.085169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 27 08:20:26.087332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:26.090163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 247/478] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 27 08:20:25 2022] POST /identity/v3/roles => generated 144 bytes in 266 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:26.090290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:26.161911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:26.205116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d37eb-202e-40c7-9d54-bef1798ffeea None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:26.243814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d606ca-f9da-4945-a412-35ae80726928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:26.244461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d606ca-f9da-4945-a412-35ae80726928 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:26.244745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d606ca-f9da-4945-a412-35ae80726928 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:26.245022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d606ca-f9da-4945-a412-35ae80726928 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:26.256295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d606ca-f9da-4945-a412-35ae80726928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-c5d606ca-f9da-4945-a412-35ae80726928 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:26.261774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:26.262731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:26.263953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 232/479] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:26.269276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:26.378034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5b7d7-b2fc-47a0-87a8-ee1216e6ff8d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:26.414161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e00166-e6d9-4072-b00a-d65b018db77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:26.414776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e00166-e6d9-4072-b00a-d65b018db77e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:26.415063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e00166-e6d9-4072-b00a-d65b018db77e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:26.415337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e00166-e6d9-4072-b00a-d65b018db77e None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:26.416068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e00166-e6d9-4072-b00a-d65b018db77e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:26.416373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e00166-e6d9-4072-b00a-d65b018db77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:26.417756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e00166-e6d9-4072-b00a-d65b018db77e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:26.419222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:26.427225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 248/480] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:27.126728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b2e4-dc32-438d-ad4f-3a85da9ae321 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:27.127275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b2e4-dc32-438d-ad4f-3a85da9ae321 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:27.127621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6b2e4-dc32-438d-ad4f-3a85da9ae321 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:27.129271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 233/481] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:27.134938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00330c-e2fa-4b8a-89d8-5a99223e9b02 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:27.135805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00330c-e2fa-4b8a-89d8-5a99223e9b02 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:27.136299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00330c-e2fa-4b8a-89d8-5a99223e9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:27.158862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-bb00330c-e2fa-4b8a-89d8-5a99223e9b02 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:27.160428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bb00330c-e2fa-4b8a-89d8-5a99223e9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:27.241149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 249/482] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:27.270935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b2e4-dc32-438d-ad4f-3a85da9ae321 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:27.400603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6b2e4-dc32-438d-ad4f-3a85da9ae321 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:27.426767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:27.440073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:27.440982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:27.444768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:27.444768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:27.452741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:27.458717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:27.459628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:27.460434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:27.463201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 234/483] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:27.468736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00330c-e2fa-4b8a-89d8-5a99223e9b02 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:27.633270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00330c-e2fa-4b8a-89d8-5a99223e9b02 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:27.671979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:27.685385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:27.686011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:27.686320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:27.686600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:27.687324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:27.687670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:27.690523 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:27.701440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 250/484] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:27.707297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:27.830598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de786-92c2-4a5c-aec4-29d97900291d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:27.870458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cb7b5-6def-414e-bf8f-5a632701ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:27.871509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb7b5-6def-414e-bf8f-5a632701ed7b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:27.872025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb7b5-6def-414e-bf8f-5a632701ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:27.872517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb7b5-6def-414e-bf8f-5a632701ed7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:27.885715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cb7b5-6def-414e-bf8f-5a632701ed7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:27.887590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cb7b5-6def-414e-bf8f-5a632701ed7b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b16cb7b5-6def-414e-bf8f-5a632701ed7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:27.904289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:27.905180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:27.906030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:27.906030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:27.906030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:27.906030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:27.908299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 235/485] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:27.913957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:27.987770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a80d9a-768e-477d-a8c9-c70280f4cce0 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:27.988068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a80d9a-768e-477d-a8c9-c70280f4cce0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:27.988374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a80d9a-768e-477d-a8c9-c70280f4cce0 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:27.989847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 236/486] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:27.995273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbcebf-9944-4628-a1d8-093e0d9b3e9c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:27.995870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbcebf-9944-4628-a1d8-093e0d9b3e9c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:27.996342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbcebf-9944-4628-a1d8-093e0d9b3e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.029181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-47cbcebf-9944-4628-a1d8-093e0d9b3e9c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:28.034466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-47cbcebf-9944-4628-a1d8-093e0d9b3e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:28.060081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9ad8e-432b-4e3d-b348-a922d922f6e8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:28.089974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:28.102840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:28.103468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.103751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.104024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.104724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:28.105028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:28.106426 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:28.119558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 251/487] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:28.124821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:28.164359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 237/488] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:20:28.198940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:28.210187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbcebf-9944-4628-a1d8-093e0d9b3e9c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:28.299189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e417b-bbee-459a-876e-4794fc77bea6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:28.364741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:28.365443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.365725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.366050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.377509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:28.378999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:28.388107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbcebf-9944-4628-a1d8-093e0d9b3e9c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:28.398978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:28.399900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:28.400794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 252/489] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:20:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:28.406711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:28.467991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:28.469780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.470069 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.470396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.493408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:28.500674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:28.502118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:28.502953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:28.505601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 238/490] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:28.514816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:28.566525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:28.609029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5b826-f508-415a-b9f1-7c7fe99175ea None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:28.628583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:28.660810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4dbeb-6767-4b60-a181-ca0bf4df386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:28.661912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4dbeb-6767-4b60-a181-ca0bf4df386c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.662481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4dbeb-6767-4b60-a181-ca0bf4df386c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.663110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4dbeb-6767-4b60-a181-ca0bf4df386c None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.664109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4dbeb-6767-4b60-a181-ca0bf4df386c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:28.664618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4dbeb-6767-4b60-a181-ca0bf4df386c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:28.666392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4dbeb-6767-4b60-a181-ca0bf4df386c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:28.678354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a2794-a71b-44f4-8458-8894851e9c87 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:28.706752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 253/491] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:20:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:28.708501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd642-af1e-405b-ba05-5f04b660a118 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.708984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd642-af1e-405b-ba05-5f04b660a118 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.709452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd642-af1e-405b-ba05-5f04b660a118 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.714283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 254/492] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:28.715440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bd642-af1e-405b-ba05-5f04b660a118 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:28.721056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc07735-ec56-4a9d-8ca9-ac3911619de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:28.721720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc07735-ec56-4a9d-8ca9-ac3911619de7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.722009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc07735-ec56-4a9d-8ca9-ac3911619de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.722312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc07735-ec56-4a9d-8ca9-ac3911619de7 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.723041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc07735-ec56-4a9d-8ca9-ac3911619de7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:28.723347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc07735-ec56-4a9d-8ca9-ac3911619de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:28.724735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc07735-ec56-4a9d-8ca9-ac3911619de7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:28.738128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 239/493] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:28.738128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849b487-776c-4a34-acf9-849c76488bb7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.738128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849b487-776c-4a34-acf9-849c76488bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.738128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849b487-776c-4a34-acf9-849c76488bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.754449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c849b487-776c-4a34-acf9-849c76488bb7 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:28.755753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c849b487-776c-4a34-acf9-849c76488bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:28.828312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 240/494] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:28.829295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849b487-776c-4a34-acf9-849c76488bb7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:28.839662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bd642-af1e-405b-ba05-5f04b660a118 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:28.849465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:28.878801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018ecaa-bac1-473e-a3d1-67f1cc270d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:28.880727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018ecaa-bac1-473e-a3d1-67f1cc270d5a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.881204 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018ecaa-bac1-473e-a3d1-67f1cc270d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.881740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018ecaa-bac1-473e-a3d1-67f1cc270d5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.918651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:28.941158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849b487-776c-4a34-acf9-849c76488bb7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:28.954540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018ecaa-bac1-473e-a3d1-67f1cc270d5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:28.956847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018ecaa-bac1-473e-a3d1-67f1cc270d5a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:28.975953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:28.976592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:28.976870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:28.977142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:28.987310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:28.988750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:29.003555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:29.004420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:29.005300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:29.005300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:29.005300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.005300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:29.005300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 241/495] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:29.005561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.061469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 255/496] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:29.062818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047c474-b1f0-4da6-b60a-c8285e2c1ae3 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.063103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047c474-b1f0-4da6-b60a-c8285e2c1ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.063375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047c474-b1f0-4da6-b60a-c8285e2c1ae3 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.064640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 256/497] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:29.065813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499d565-a526-4ad0-bd55-82273e53bb20 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.066090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499d565-a526-4ad0-bd55-82273e53bb20 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.066526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499d565-a526-4ad0-bd55-82273e53bb20 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.070188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 257/498] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:29.070188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499d565-a526-4ad0-bd55-82273e53bb20 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.077574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.112841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1beb-3e7a-45be-af7b-eecd6f300af7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.137391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.147157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.147852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.148138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.151153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.159306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:29.164730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:29.166174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 242/499] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:29.167007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.175806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499d565-a526-4ad0-bd55-82273e53bb20 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.210286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e566029-c092-4389-ae75-d701e90d58fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.211192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566029-c092-4389-ae75-d701e90d58fd None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.211488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566029-c092-4389-ae75-d701e90d58fd None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.211763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e566029-c092-4389-ae75-d701e90d58fd None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.212455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e566029-c092-4389-ae75-d701e90d58fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:29.212758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e566029-c092-4389-ae75-d701e90d58fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.214175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e566029-c092-4389-ae75-d701e90d58fd None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.225739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 258/500] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:29.227274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da1f0b-b0fd-4619-af12-c9f7f7f56fde None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.227558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da1f0b-b0fd-4619-af12-c9f7f7f56fde None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.227831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da1f0b-b0fd-4619-af12-c9f7f7f56fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.245853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e3da1f0b-b0fd-4619-af12-c9f7f7f56fde None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:29.247100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e3da1f0b-b0fd-4619-af12-c9f7f7f56fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:29.267854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1d8d4-9fb8-42dc-98d0-84463ad16e66 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.295145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.303137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.303762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.304039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.305485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.311893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 259/501] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:29.313115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f077cb9-c14a-49a2-99d8-00d37ac92416 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.313402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f077cb9-c14a-49a2-99d8-00d37ac92416 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.313686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f077cb9-c14a-49a2-99d8-00d37ac92416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.325817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.327595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.331505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9f077cb9-c14a-49a2-99d8-00d37ac92416 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:29.332696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9f077cb9-c14a-49a2-99d8-00d37ac92416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:29.344148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:29.345033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:29.346556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:29.346556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:29.346556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.346556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:29.346556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 243/502] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:29.346976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.386826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.403024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 260/503] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:29.403897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f077cb9-c14a-49a2-99d8-00d37ac92416 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.461713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c415-a9c4-4767-8f1c-f7edca8daad7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.494061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.494681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.494959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.495231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.495932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:29.496235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.497586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.499950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f077cb9-c14a-49a2-99d8-00d37ac92416 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.505797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 244/504] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:29.506603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.536872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.537504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.537781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.538054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.545610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.547094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.549864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:29.562058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:29.567264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 261/505] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:29.568193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.612037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839f167-70bc-4d31-a966-e24491f69a1c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.613533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.645623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.646211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.646488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.646760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.656778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:29.662367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:29.663257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:29.664227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 245/506] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:29.665627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.668606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e859a3-da21-4447-90f9-a0fb6b86c31c None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.702028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.702666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.702940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.703208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.703890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:29.704189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.706089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.717555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 262/507] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:29.718362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.767561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18612821-33ae-404f-81c0-efc246fb5c71 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.769266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.801706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.802385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.802667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.803054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.814917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.816368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.816856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ac4e4-75ec-4369-8aad-4c58df7a6d30 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.824117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:29.832152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:29.838698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:29.839640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:29.839640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:29.839640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.839640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:29.839640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 246/508] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:29.839640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.852857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.853486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.853766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.854038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.864637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.866115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:29.881402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:29.882380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:29.883460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:29.883460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:29.883460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:29.883460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:29.883460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 263/509] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:29.883810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:29.938414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e371c9-26fa-49b6-a2b6-10885c56f002 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:29.972352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:29.972960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:29.973236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:29.973513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:29.974319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:29.974623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:29.976020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:29.982688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6001b43-74c9-4b48-8339-f9cec54c2447 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.012118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 247/510] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:30.012943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.015550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.016164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.016442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.016780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.018115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:30.018457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.019830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.027866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 264/511] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:30.028496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.028650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.049859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.111824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d27cc-3548-4480-ae5f-aeed179f7c8f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.126851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39045eb-c617-4211-85ba-4e978ab1942b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.146064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.146728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.147008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.147277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.154243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.155698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.159743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.160340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.160617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.160887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.161569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:30.161868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.163244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:30.167424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:30.168612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:30.169469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 248/512] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:30.169945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.178917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 265/513] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:30.179756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.239453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.260341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.269186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabec6e0-63ea-43d3-850a-11249c901719 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.278858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1616146-b24d-438f-a1bb-6e0a093fe3f8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.303249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.303853 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.304148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.304424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.305111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:30.305419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.306851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.312402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.313003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.313280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.313556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.320270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 249/514] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:30.321069 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.383267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.384859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.417556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca9ef8-e03d-46c4-b11f-e0f84606fdb9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.448185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.453034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.453652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.455038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.455038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.463934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.464684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.465353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:30.480325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:30.481328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:30.483784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:30.483784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:30.483784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:30.483784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:30.483784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 250/515] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:30.483784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.494634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 266/516] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:30.494634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.583102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cfc11-090c-4ea4-9241-a3a26cb6b594 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.591172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4324a2a-3a22-4b81-b2e0-ffe0f9de62e1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.617289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.617906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.618205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.618482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.619226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:30.619529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.620960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.625012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.625621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.625901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.626201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.627154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.628660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.638339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 251/517] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:30.639159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.685627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.701882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 267/518] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:20:30 2022] POST /identity/v3/users => generated 247 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:30.702732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.744015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.753701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec402e1-b1ce-4cb3-9771-c34d534e8cae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.794300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.795280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.795560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.795833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.804219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.805704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.819677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4774e-cac8-472d-a998-41d2dd5d68de None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:30.820220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:30.821243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:20:30.822383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:30.826494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 252/519] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:30.827481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.863949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.864833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.865117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.865389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.866086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:30.866420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.867796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.881253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 268/520] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:30.882291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:30.944194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03939f5c-1612-4be5-afeb-3480a4a39793 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:30.946726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.980008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:30.980797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:30.981083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:30.981357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:30.991577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:30.991755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:30.993268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:30.998808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7d3d8-3af4-4062-8ae2-395bd02cbc52 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:31.009759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:31.010701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:31.011732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:31.011732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:31.011732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:31.011732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:31.011732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 253/521] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:31.015963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.039525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.040289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.040663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.041028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.041957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e4e461b0b6142b38fd7de56395733b2) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:31.042397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.043924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.058221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 269/522] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:30 2022] GET /identity/v3/role_assignments?user.id=4e4e461b0b6142b38fd7de56395733b2&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:31.058221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.149197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747bf3-b642-451a-8c79-13e4c765ff49 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.188281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.189461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.189857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.190229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.191358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:31.191774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.193522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.202537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40aaff-d242-4b8e-9d5d-3c875b93a6ff None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.215897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 254/523] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:31.216756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.234601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:31.253878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.254553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.254829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.255106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.264528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.267788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:31.284550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:31.285464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 08:20:31.286485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:31.290215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 270/524] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:31.291425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.326593 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:31.334906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3e54d-fb71-4c91-936c-5a54a7a31dbc None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.371904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.372615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.372956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.373312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.374073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:31.374474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.376018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.394375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 255/525] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:31.395410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.422212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991be147-3901-4aea-a009-0dd2b734f394 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.457823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.458488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.458772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.459044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.497101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:31.529886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d115bc78-c268-440e-832d-c2150e8dc395 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.533191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.534873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.560712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:31.579597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.580217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.580499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.580777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.581575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:31.581879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.583792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.608746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 256/526] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:31.611165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.662365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 271/527] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:31.663687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.719916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b38e5-99c8-476a-a8ba-a0b5b829ca1f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.750508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:31.756327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.764625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.764625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.764625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.764625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.766172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.772196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e309fd-7f9c-444f-abd3-bd83f63c2726 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:31.778786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:31.779998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:31.780871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:31.784697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 257/528] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:31.785567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:31.799977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:31.817735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.818482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.818767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.819037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.895935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.897676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.900540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959796c6-aa6b-472f-bc8f-e2b093f59244 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:31.936788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:31.938532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:31.939864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:31.940631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:31.954015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:31.955789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:31.970391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:31.971498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:31.972337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:31.972337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:31.972337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:31.972337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:31.974490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 258/529] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:31.975480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.013029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 272/530] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:32.013029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.027077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:32.033783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:32.083850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21a4d-bb40-4949-96d7-d4301b1f98cb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.117689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e9421-03fb-4c80-932e-9fbedbb86139 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.119004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.119839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.120318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.120811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.121769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:32.122297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.123925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.142399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 259/531] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:32.143654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.156082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.156841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.157329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.158194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.158925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:32.159239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.160618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.172929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 273/532] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:32.174414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.250786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ae735-7f23-496c-b48a-a521a4b8567f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.261988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:32.262569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:32.282290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ecdfb-d48e-453f-9748-f8b6ef9707a4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.286697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.287284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.287567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.287844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.298339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.299842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:32.315385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:32.321510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:32.322414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:32.322414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:32.322414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:32.322414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:32.322414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 260/533] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:32.322414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.324911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.325590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.325935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.326296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.327153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e41212825ad4990958a7e909c669177) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:32.327528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.329094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.336362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 274/534] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/role_assignments?user.id=4e41212825ad4990958a7e909c669177&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:32.337270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.506657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc99f82-1926-40d8-8c18-c2d7cb19beb6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.547970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12fd6-8f60-4f3d-be03-aaa62b79bb95 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.562903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.563569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.563917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.564202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.583326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.584811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.592863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.593486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.593766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.594039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.594755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:32.595179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.597210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.610497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 275/535] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:32.610960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:32.612903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:32.615106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:32.616053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 261/536] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:32.620613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.620613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:32.621854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.656378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:32.722625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71beec1-c8da-4c9d-983d-334bb1728730 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.758717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.759462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.759816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.760161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.768085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.769629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.780793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb25a2b-dbe2-42c9-9987-958e7d968ac7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:32.782303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:32.784327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:32.786196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 276/537] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:32.790376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.859131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:32.898173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10f445-c67c-4a0c-a8e2-c53e497e9bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.898828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10f445-c67c-4a0c-a8e2-c53e497e9bee None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.899109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10f445-c67c-4a0c-a8e2-c53e497e9bee None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.899380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10f445-c67c-4a0c-a8e2-c53e497e9bee None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.900133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10f445-c67c-4a0c-a8e2-c53e497e9bee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:32.900436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10f445-c67c-4a0c-a8e2-c53e497e9bee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.901130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc17c0-a150-408b-8a7c-e1a524a124d2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:32.901881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10f445-c67c-4a0c-a8e2-c53e497e9bee None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.929470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 262/538] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:32.931076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25de7-9256-4a49-898d-186aaac3d84b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.931362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25de7-9256-4a49-898d-186aaac3d84b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.931634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda25de7-9256-4a49-898d-186aaac3d84b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.934979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 263/539] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:32.937620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25de7-9256-4a49-898d-186aaac3d84b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:32.938665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4dd4c-3535-47c4-b4ed-155ba306f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:32.939937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4dd4c-3535-47c4-b4ed-155ba306f615 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.940226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4dd4c-3535-47c4-b4ed-155ba306f615 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.940497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4dd4c-3535-47c4-b4ed-155ba306f615 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.941442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4dd4c-3535-47c4-b4ed-155ba306f615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:32.941762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4dd4c-3535-47c4-b4ed-155ba306f615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:32.944127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4dd4c-3535-47c4-b4ed-155ba306f615 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:32.967246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 277/540] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:32.971334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518e8a4-3f59-4e30-944d-cf0a8714bd82 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:32.971619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518e8a4-3f59-4e30-944d-cf0a8714bd82 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:32.971889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518e8a4-3f59-4e30-944d-cf0a8714bd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:32.990692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9518e8a4-3f59-4e30-944d-cf0a8714bd82 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:32.991999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9518e8a4-3f59-4e30-944d-cf0a8714bd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:33.037146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:33.062293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda25de7-9256-4a49-898d-186aaac3d84b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:33.082474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 278/541] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:33.083386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518e8a4-3f59-4e30-944d-cf0a8714bd82 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:33.099977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:33.100693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.101048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.101393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.108570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=99a29a23f8a84076bab3adefc042a088) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:33.108933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:33.110500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:33.120918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 264/542] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:32 2022] GET /identity/v3/role_assignments?user.id=99a29a23f8a84076bab3adefc042a088&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:33.121906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:33.168882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:33.232194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab368f28-dbed-4bb2-90f3-7100a87c1b94 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:33.241594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518e8a4-3f59-4e30-944d-cf0a8714bd82 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:33.269900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67c381-1559-4d47-bcef-7db7f51f1f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:33.270614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67c381-1559-4d47-bcef-7db7f51f1f56 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.270855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67c381-1559-4d47-bcef-7db7f51f1f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.271129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67c381-1559-4d47-bcef-7db7f51f1f56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.279832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:33.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67c381-1559-4d47-bcef-7db7f51f1f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:33.285081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c964637-1153-4c8d-bfb6-2b151aaf6d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:33.286357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c964637-1153-4c8d-bfb6-2b151aaf6d2d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.286846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c964637-1153-4c8d-bfb6-2b151aaf6d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.287348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c964637-1153-4c8d-bfb6-2b151aaf6d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.288342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c964637-1153-4c8d-bfb6-2b151aaf6d2d None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=d5c308c33c674f1ea7111c0297174c2f) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:33.288829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c964637-1153-4c8d-bfb6-2b151aaf6d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-5a67c381-1559-4d47-bcef-7db7f51f1f56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:33.289569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:33.297749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:33.299143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 265/543] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:33.299797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c964637-1153-4c8d-bfb6-2b151aaf6d2d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:33.299868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed25ac-6e72-4125-8d02-55bba3a6fb0f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.299868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed25ac-6e72-4125-8d02-55bba3a6fb0f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.299868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed25ac-6e72-4125-8d02-55bba3a6fb0f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.299868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 266/544] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:33.304618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed25ac-6e72-4125-8d02-55bba3a6fb0f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:33.305761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 279/545] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/role_assignments?user.id=d5c308c33c674f1ea7111c0297174c2f&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:33.307440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3202b-f508-4046-8cff-41ee31939850 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.307958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3202b-f508-4046-8cff-41ee31939850 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.308424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3202b-f508-4046-8cff-41ee31939850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.347533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-55e3202b-f508-4046-8cff-41ee31939850 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:33.349205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-55e3202b-f508-4046-8cff-41ee31939850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:33.457149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 280/546] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:33.469956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed25ac-6e72-4125-8d02-55bba3a6fb0f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:33.492718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3202b-f508-4046-8cff-41ee31939850 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:33.540120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:33.540746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.541049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.541319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.542031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:33.542575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:33.543938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:33.560559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 267/547] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:33.573639 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:33.588992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:33.651961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3202b-f508-4046-8cff-41ee31939850 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:33.701424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:33.702657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.702943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.703215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.712180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:33.729138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:33.737156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:33.738123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:33.738942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:33.744734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 281/548] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:33.747635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:33.779494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1d58a-a1a6-45eb-b770-80e6d96a526c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:33.847753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:33.848395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.848680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.848955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.876655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:33.878353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:33.888442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752d03b-06d1-4d57-8491-b6f8a2744c36 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:33.903798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:33.907084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:33.908374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:33.909233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:33.909233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:33.909233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:33.909233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:33.916204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 268/549] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:33.925666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:33.959409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:33.960404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:33.960923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:33.961392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:33.962429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:33.962930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:33.964558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:33.977685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 282/550] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:33.987770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:34.058169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d05580-eb44-4a3d-9edc-d0a09488e854 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:34.145289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:34.171293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:34.174489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.175101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.175827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.177699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:34.178231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:34.181855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:34.211482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 269/551] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:34.217034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:34.263220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0599adb-85e9-4b07-80d3-235a49ed2acb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:34.336584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:34.339059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.339703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.340966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.372376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:34.373617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:34.399124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ba842-1339-4b36-b894-047b509f5a30 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:34.404968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:34.405841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:34.406735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:34.406735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:34.406735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:34.406735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:34.414580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 283/552] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:34.423185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:34.447389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:34.448464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.448981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.449451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.465984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:34.468850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:34.490410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:34.491446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:34.492337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:34.493844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:34.496033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 270/553] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:34.503523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:34.552038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:34.611708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b456683-37b7-492f-95c8-1e26834fa061 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:34.628978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8d34-bab8-4c75-929c-4c69d43e0774 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:34.653021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d7238-d741-4704-b4b7-5a6eba8a57de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:34.653947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d7238-d741-4704-b4b7-5a6eba8a57de None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.654505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d7238-d741-4704-b4b7-5a6eba8a57de None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.654977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d7238-d741-4704-b4b7-5a6eba8a57de None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.655906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d7238-d741-4704-b4b7-5a6eba8a57de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:34.656428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d7238-d741-4704-b4b7-5a6eba8a57de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:34.658040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d7238-d741-4704-b4b7-5a6eba8a57de None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:34.668757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf51470-55de-4e96-8d80-6f6e55461831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:34.669380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51470-55de-4e96-8d80-6f6e55461831 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.669660 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51470-55de-4e96-8d80-6f6e55461831 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.669932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51470-55de-4e96-8d80-6f6e55461831 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.670776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51470-55de-4e96-8d80-6f6e55461831 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:34.671082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51470-55de-4e96-8d80-6f6e55461831 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:34.672097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 284/554] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:34.672492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51470-55de-4e96-8d80-6f6e55461831 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:34.674137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0cb6-af03-4b71-a899-69962fcd273b None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.674649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0cb6-af03-4b71-a899-69962fcd273b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.675175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db0cb6-af03-4b71-a899-69962fcd273b None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.677274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 285/555] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:34.678342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db0cb6-af03-4b71-a899-69962fcd273b None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:34.696648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 271/556] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 27 08:20:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:34.696648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bde5dc-1704-4f00-aa3c-17e248de8154 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.696648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bde5dc-1704-4f00-aa3c-17e248de8154 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.696648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bde5dc-1704-4f00-aa3c-17e248de8154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.714017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-87bde5dc-1704-4f00-aa3c-17e248de8154 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:34.715261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-87bde5dc-1704-4f00-aa3c-17e248de8154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:34.786272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 272/557] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:34.787061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bde5dc-1704-4f00-aa3c-17e248de8154 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:34.810515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:34.819739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:34.834916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db0cb6-af03-4b71-a899-69962fcd273b None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:34.873191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:34.874165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.874586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.876871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:34.883934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:34.885712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:34.898654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:34.900237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:34.901133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:34.901693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bde5dc-1704-4f00-aa3c-17e248de8154 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:34.904186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 286/558] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:20:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:34.905278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:34.936926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:34.937647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:34.937995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:34.938371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.008989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.010532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.019387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c1781-edad-4d33-a88b-659b39520600 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.035881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.059103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.060128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.060607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.061051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.073389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:35.080218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:35.083691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:35.084555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 287/559] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:35.086452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.089568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.117215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 273/560] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:35.118031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.231149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163d18d-4b60-4ccc-be14-2ce6379fcf3b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.238995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271d0c8-bf11-478b-9bda-6fc3c1871a7e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.270096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.273905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.274569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.274848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.275123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.276098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:35.276711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.278465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.285827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.287143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.287595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.290157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.290157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:35.290660 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.292235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.298483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 288/561] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:35.300074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.303792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 274/562] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:35.305872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.371407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.418762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8c1e-c9f9-4d7e-b70c-d4764a4b1595 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.420569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b669b1-5c97-48e0-81e6-b6853f4a5a94 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.456849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.457245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a92ce-7b84-4292-91b9-bc9b71c26b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.457486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.457765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.457888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a92ce-7b84-4292-91b9-bc9b71c26b20 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.458046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.458232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a92ce-7b84-4292-91b9-bc9b71c26b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.458435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a92ce-7b84-4292-91b9-bc9b71c26b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.468701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a92ce-7b84-4292-91b9-bc9b71c26b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.469217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.470226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a92ce-7b84-4292-91b9-bc9b71c26b20 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.471101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-6a5a92ce-7b84-4292-91b9-bc9b71c26b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:35.485907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:35.486983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:35.496918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:35.496918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:35.496918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:35.496918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:35.497148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:35.497909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:35.498803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:35.498803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:35.498803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:35.498803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:35.498803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 275/563] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:35.499066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 289/564] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:35.499140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.499209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c7833-abcc-4ddb-9c57-a0ebe5d1984f None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.499209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c7833-abcc-4ddb-9c57-a0ebe5d1984f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.499209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c7833-abcc-4ddb-9c57-a0ebe5d1984f None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.499209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 290/565] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:35.499209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c7833-abcc-4ddb-9c57-a0ebe5d1984f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.535051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.608452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1a6d3-ae4a-4386-9d04-1549b945fdb4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.609915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c7833-abcc-4ddb-9c57-a0ebe5d1984f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.624445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.644052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954c51d-f346-4b70-928c-2f6ef7604dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.644781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c51d-f346-4b70-928c-2f6ef7604dbc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.645073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c51d-f346-4b70-928c-2f6ef7604dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.645213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.645351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954c51d-f346-4b70-928c-2f6ef7604dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.645820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.646098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.646393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.647086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:35.647384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.648760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.656096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954c51d-f346-4b70-928c-2f6ef7604dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.657609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954c51d-f346-4b70-928c-2f6ef7604dbc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.664854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 291/566] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:35.664854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b954c51d-f346-4b70-928c-2f6ef7604dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:35.672201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:35.673912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:35.675050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:35.675050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:35.675050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:35.675050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:35.675050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 276/567] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:35.675399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36949184-12b5-49de-9571-3028147cfc57 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.675399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36949184-12b5-49de-9571-3028147cfc57 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.675644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36949184-12b5-49de-9571-3028147cfc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.694306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-36949184-12b5-49de-9571-3028147cfc57 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:35.695540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-36949184-12b5-49de-9571-3028147cfc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:35.762047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 277/568] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:35.762568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bb7d4-67cb-4f3d-9004-19ad2d3b6eba None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.763742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d62a62-673f-4eee-9695-71160b001c11 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.764037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d62a62-673f-4eee-9695-71160b001c11 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.764310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d62a62-673f-4eee-9695-71160b001c11 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.765762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 278/569] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:35.766456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d62a62-673f-4eee-9695-71160b001c11 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.801378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.803219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.803832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.804121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.804392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.805080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:35.805383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.806782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.817989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 292/570] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:35.818821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:35.851281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:35.886717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d62a62-673f-4eee-9695-71160b001c11 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.919002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492f6c8-5b61-4f65-95be-b084708fb133 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:35.920347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4db2c-3787-4aa0-8d79-e112f1c064ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.920974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4db2c-3787-4aa0-8d79-e112f1c064ce None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.921256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4db2c-3787-4aa0-8d79-e112f1c064ce None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.921525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4db2c-3787-4aa0-8d79-e112f1c064ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.928149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4db2c-3787-4aa0-8d79-e112f1c064ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.929578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4db2c-3787-4aa0-8d79-e112f1c064ce None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-97b4db2c-3787-4aa0-8d79-e112f1c064ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:35.941018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:35.941990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:35.942921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 279/571] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:35.943401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723a572-455b-4e42-afdc-0afda82f9239 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.943625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723a572-455b-4e42-afdc-0afda82f9239 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.943835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723a572-455b-4e42-afdc-0afda82f9239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.955215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:35.955935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:35.956209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:35.956483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:35.957176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:35.957481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:35.958910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:35.961225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9723a572-455b-4e42-afdc-0afda82f9239 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:35.962457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9723a572-455b-4e42-afdc-0afda82f9239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:35.970921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 293/572] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:35.971718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.029547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 280/573] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:36.030698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723a572-455b-4e42-afdc-0afda82f9239 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.049650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.064783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.073308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d7c6-43ae-44e2-8068-a7caebb507a8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.106722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.107352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.107629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.107899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.117825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:36.122702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:36.123956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 294/574] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:36.125020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.141711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723a572-455b-4e42-afdc-0afda82f9239 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.176082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.176717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.176998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.177272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.183897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.185324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:36.196804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:36.197871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:36.198828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 281/575] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:36.199319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.226869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389e0d-b0a4-4637-af7b-12cbaa32aeb4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.260498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.261108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.261392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.261700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.262548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:36.262858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.264302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.270555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.277714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 295/576] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:36.278558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.293364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.303719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190adcf-3c65-4a82-96ef-89b3813e1d78 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.338683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.339314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.339585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.339853 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.347158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.348581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:36.361426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:36.362373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 282/577] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:36.363304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.383134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7182e64-5bce-4084-938c-2dc24b6ac158 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.415627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.416177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.416451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.416729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.427624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:36.432486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:36.433588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:36.434592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 296/578] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:36.435267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.460275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb81fdf-6c23-4e2b-80f0-24df74d14e4d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.486580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.493942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.494584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.499375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.499375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.499375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:36.499375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.499375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.505991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.506266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 283/579] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:36.507033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.534257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3315-22e4-4d4d-a0bf-2ef3b6bbfa3a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.570744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.571354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.571632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.571903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.572645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:36.572959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.574406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.585910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 297/580] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:36.586744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.609987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8966-0389-4012-8044-cc81cb1c5b87 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.645494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.646176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.646459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.646732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e4e461b0b6142b38fd7de56395733b2/roles/b2f748f19ea54f688475e569b0847642` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.661144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=4e4e461b0b6142b38fd7de56395733b2, role_id=b2f748f19ea54f688475e569b0847642)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.662621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.684398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c923c3-b52a-4a38-a517-c2641a29020d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.711615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.716290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.716895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.717174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.717446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.718257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:36.718563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.720948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.731043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 284/581] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:20:36 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e4e461b0b6142b38fd7de56395733b2/roles/b2f748f19ea54f688475e569b0847642 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:36.731862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.738768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 298/582] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:36.739646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.791875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:36.853755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3be4b6-9307-4daf-8faf-12619e5a9a16 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.858705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922e89b-0d79-46cd-a6ab-82195e4c3faa None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:36.897616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.898580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.899439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.899439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.899624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:36.900140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:36.900421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:36.900692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:36.901440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:36.901751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.903165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.911372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:36.916033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 285/583] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:36.916161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:36.916239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:36.932638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:36.933563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:36.934814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:36.934814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:36.934814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:36.934814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:36.934814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 299/584] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:36.935096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:36.981267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.029072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b8de7-7064-4bc3-b355-d5ebbf55856b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.030668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.043802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cfd73-880b-4c24-bbc0-40f35fa60b2e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.065246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.065923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.066222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.066491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.067277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=92173e230c3e45c3b769221fb4b3d666) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:37.067583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.069002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.074553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 286/585] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/role_assignments?user.id=92173e230c3e45c3b769221fb4b3d666&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:37.075355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.089310 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c907-5978-43a2-be84-153d76518500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.089968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c907-5978-43a2-be84-153d76518500 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.090294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c907-5978-43a2-be84-153d76518500 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.090640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919c907-5978-43a2-be84-153d76518500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.102887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919c907-5978-43a2-be84-153d76518500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.104375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919c907-5978-43a2-be84-153d76518500 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-d919c907-5978-43a2-be84-153d76518500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:37.120382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:37.121551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:37.122715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:37.122715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:37.122715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:37.122715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:37.127313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 300/586] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:37.129771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c907-5978-43a2-be84-153d76518500 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.223182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853bd5c-b59c-4486-9351-95e184688873 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.237786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.244879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919c907-5978-43a2-be84-153d76518500 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.271136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.272314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.272744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.273984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.286129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.287053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.287429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.287797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.288592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:37.288945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.289062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.290435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.290816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.304686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 301/587] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:37.306027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:37.312330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:37.313474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:37.314921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:37.314921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:37.314921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:37.314921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:37.314921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 287/588] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:37.315855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.339927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.427268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa35356-8836-42fc-8414-1beeda569db3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.456049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57b23e-1999-41d2-bcc5-a077f243f6e9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.463869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175d017-86d0-4309-a862-93c905537810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.464728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d017-86d0-4309-a862-93c905537810 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.465228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d017-86d0-4309-a862-93c905537810 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.465708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175d017-86d0-4309-a862-93c905537810 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.466631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175d017-86d0-4309-a862-93c905537810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:37.467132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175d017-86d0-4309-a862-93c905537810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.468702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175d017-86d0-4309-a862-93c905537810 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.485076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 302/589] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:37.486330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175d017-86d0-4309-a862-93c905537810 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.497863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.498561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.498846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.499122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.510560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.510881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.512693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:37.528778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:37.529859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 288/590] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:37.537994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.606518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175d017-86d0-4309-a862-93c905537810 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.627859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.641013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.642278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.642586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.642825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.645794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:37.645794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.645794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.652879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe3434-2a9b-4a4f-b50a-bf0a29c9391f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.657058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 303/591] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:37.657974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.690800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.691527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.691903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.692250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.699531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.701050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:37.713821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:37.714757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:37.715830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 289/592] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:37.717178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.754649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.762808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba7594-9602-4a6e-8df8-ed7b6454f85a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.799878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.800592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.800878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.801155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.801958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:37.802307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.803757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.825017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 304/593] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:37.825856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.845921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91681826-51db-4321-a90d-1204e2a854c7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.888021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:37.888694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:37.889311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:37.889585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:37.904745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:37.906495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:37.909603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:37.922615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:37.923706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 290/594] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:37.925295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:37.962391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a89bcf-d232-42f9-a8de-0545fd560479 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:37.994578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:37.999673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.000334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.000697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.000921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.002502 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:38.006291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.006291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.018805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 305/595] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:38.019632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.068296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7433f1-0182-4ab9-9ac6-88ca78ff4642 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.114999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.115963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.116920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.117781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.129394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517e684-75f1-4f81-b050-f7147e5c168b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.133420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.135108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:38.153361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:38.154288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:38.156081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:38.156081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:38.156081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:38.156081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:38.156081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 291/596] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:38.156081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.175200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.176431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.177366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.177957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.179036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:38.179728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.181700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.205731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 306/597] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 27 08:20:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:38.207522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.221008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:38.269747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:38.281881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b3b7d-b0af-4a51-bf1b-014563f73a8c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.318647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.319585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.320849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.321268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e41212825ad4990958a7e909c669177/roles/b2f748f19ea54f688475e569b0847642` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.322921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a1ed7-6526-45c4-a3dd-11a8235d9bfc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.338261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=4e41212825ad4990958a7e909c669177, role_id=b2f748f19ea54f688475e569b0847642)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.339993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.361079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.361859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.362276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.362650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.363562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:38.363963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.365420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.377937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 307/598] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:38.378702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.409725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 292/599] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:20:38 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e41212825ad4990958a7e909c669177/roles/b2f748f19ea54f688475e569b0847642 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:38.411641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.544901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f49363-ed99-4f91-842c-fb1bd98af04d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.546274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:38.552658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df6d2c-e878-4d90-b5a9-516fe571bee2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.573773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:38.597118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.598090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.598989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.599497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.609960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.612580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:38.628089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:38.629041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:38.630103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:38.632128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 293/600] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:38.639206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.646671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.647353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.647604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.647963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.667596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.670888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:38.685956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:38.686908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:38.687777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:38.687777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:38.687777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:38.687777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:38.688469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 308/601] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:38.693068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.756488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeec80e-9641-4533-b7bf-aef02597c299 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.793815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.794863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.795421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.795894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.796936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:38.797438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.799287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.812461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 294/602] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:38.819096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.850381 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a5505-4c7c-401c-8adf-a9206d7c4c01 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.851289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:38.903638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe4985-44f7-4c3e-a6f7-ad46272094af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.904258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4985-44f7-4c3e-a6f7-ad46272094af None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.904542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4985-44f7-4c3e-a6f7-ad46272094af None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.904818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4985-44f7-4c3e-a6f7-ad46272094af None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.905678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe4985-44f7-4c3e-a6f7-ad46272094af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:38.905996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe4985-44f7-4c3e-a6f7-ad46272094af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:38.908115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe4985-44f7-4c3e-a6f7-ad46272094af None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:38.921421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 309/603] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:38.923005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56598904-11a7-4d15-89fc-9aee4a54ed9d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.923630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56598904-11a7-4d15-89fc-9aee4a54ed9d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.923919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56598904-11a7-4d15-89fc-9aee4a54ed9d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:38.925661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 310/604] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:38.926467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56598904-11a7-4d15-89fc-9aee4a54ed9d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:38.939389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d6050-ae2d-4e9e-9eab-084dd730caee None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:38.953560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:38.984649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:38.986507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:38.987003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:38.988445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/99a29a23f8a84076bab3adefc042a088/roles/b2f748f19ea54f688475e569b0847642` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.012730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=99a29a23f8a84076bab3adefc042a088, role_id=b2f748f19ea54f688475e569b0847642)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:39.015469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:39.042350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56598904-11a7-4d15-89fc-9aee4a54ed9d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:39.078857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f250d-fc68-409f-a764-5b2814e4d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:39.079581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f250d-fc68-409f-a764-5b2814e4d2fe None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.079939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f250d-fc68-409f-a764-5b2814e4d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.080213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f250d-fc68-409f-a764-5b2814e4d2fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.087254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f250d-fc68-409f-a764-5b2814e4d2fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:39.088869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f250d-fc68-409f-a764-5b2814e4d2fe None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-be6f250d-fc68-409f-a764-5b2814e4d2fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:39.104124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:39.105041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 311/606] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a08af-8a4b-4cfe-9384-f8ef600ff11d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.105939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a08af-8a4b-4cfe-9384-f8ef600ff11d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.106571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a08af-8a4b-4cfe-9384-f8ef600ff11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.107370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 295/606] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:20:38 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/99a29a23f8a84076bab3adefc042a088/roles/b2f748f19ea54f688475e569b0847642 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:39.109146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:39.143608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-450a08af-8a4b-4cfe-9384-f8ef600ff11d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:39.144933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-450a08af-8a4b-4cfe-9384-f8ef600ff11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:39.234548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:39.241696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 312/607] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:39.252460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770c7e5-ad95-47d7-8bf3-32a25cdbc5bb None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.252460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770c7e5-ad95-47d7-8bf3-32a25cdbc5bb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.252460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770c7e5-ad95-47d7-8bf3-32a25cdbc5bb None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.252460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 313/608] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:39.252460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8118e-ee3c-4ffb-b8ad-4403af548182 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.252460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8118e-ee3c-4ffb-b8ad-4403af548182 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.252904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8118e-ee3c-4ffb-b8ad-4403af548182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.255034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa7af4-5398-43d3-b261-35875a9a01bd None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:39.281567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-14a8118e-ee3c-4ffb-b8ad-4403af548182 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:39.284186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-14a8118e-ee3c-4ffb-b8ad-4403af548182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:39.292542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:39.296150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:39.299171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.299171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.299171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.299171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:39.299477 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:39.300779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:39.313778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 296/609] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:39.315120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:39.372313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 314/610] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:20:39.373150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8118e-ee3c-4ffb-b8ad-4403af548182 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:39.452642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe288f-a879-4e37-b97b-40736a78bac1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:39.494391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:39.503513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:39.504412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.504924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.505050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8118e-ee3c-4ffb-b8ad-4403af548182 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:39.505449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.520264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:39.526495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:39.527451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:39.528301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 297/611] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:39.529029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:39.552407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:39.553111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.553395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.553674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/d5c308c33c674f1ea7111c0297174c2f/roles/b2f748f19ea54f688475e569b0847642` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.577583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:39.581200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=d5c308c33c674f1ea7111c0297174c2f, role_id=b2f748f19ea54f688475e569b0847642)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:39.582914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:39.646850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0eea-b6c2-4a20-8e49-f2a32342a226 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:39.682001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 315/612] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:20:39 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/d5c308c33c674f1ea7111c0297174c2f/roles/b2f748f19ea54f688475e569b0847642 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:39.684055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:39.696222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:39.696912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.698237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.698237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.723509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:39.736684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:39.737684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:39.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:39.743754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 298/613] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:39.746668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:39.853281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:39.908090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405be0a2-9e9b-449d-bf00-8ae83c4d6174 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:39.942228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335790-8a1b-4bd4-a887-ac22e4d1ab51 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:39.955100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:39.955688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.955972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.956240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.956959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:39.957285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:39.958659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:39.967586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 316/614] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:39.973926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:39.989857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:39.990843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:39.991228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:39.991942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:39.992764 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:39.993204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:39.994726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:40.004942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 299/615] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:40.010840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:40.123817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ac62f-0dbd-4846-b5c7-cc3013c396b7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:40.128267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:40.154600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:40.162310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:40.163337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.163724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.164100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.177627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:40.179258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:40.195631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:40.196712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:40.197783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:40.197783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:40.197783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:40.197783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:40.200162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 300/616] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:40.208447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:40.259231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a77527-f2d2-48e8-934b-5fd141d8391f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:40.309326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:40.310124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.310542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.310917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.324024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:40.325626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:40.342685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:40.343854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:40.345038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:40.345038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:40.345038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:40.345038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:40.347401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 317/617] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:40.352638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:40.445150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50d75c-e59e-4abc-a5ad-f2bd9a93574e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:40.459040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e338bc-5bb0-4c7b-8563-27d865730a47 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:40.477109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:40.499363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:40.499986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.500268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.500542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.501833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:40.502135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:40.503513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:40.505533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:40.506154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.506433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.506700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.507384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:40.507685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:40.509034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:40.529438 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 318/618] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:40.536570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:40.537188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 301/619] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:40.543943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:40.597089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:40.726877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a92ee-cfcb-4a3f-8d72-9c7c0125426b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:40.743075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c3f61-c5ce-442b-a992-b59534cc32db None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:40.792770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09929862-da95-442b-bad4-62b323fa2c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:40.793406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09929862-da95-442b-bad4-62b323fa2c97 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.793684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09929862-da95-442b-bad4-62b323fa2c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.793952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09929862-da95-442b-bad4-62b323fa2c97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.806810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09929862-da95-442b-bad4-62b323fa2c97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:40.808268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09929862-da95-442b-bad4-62b323fa2c97 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:40.812804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:40.813994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.814352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.814624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.823269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:40.825109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-09929862-da95-442b-bad4-62b323fa2c97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:40.827657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:40.828602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:40.829645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 319/620] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:20:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:40.834694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1c129-f985-4de6-b131-94d2bca05aed None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.834694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1c129-f985-4de6-b131-94d2bca05aed None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.834694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1c129-f985-4de6-b131-94d2bca05aed None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.836508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 320/621] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:40.838076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b874e2-cad7-4357-853a-7426a56f5165 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:40.838483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b874e2-cad7-4357-853a-7426a56f5165 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:40.838831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b874e2-cad7-4357-853a-7426a56f5165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:40.846358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:40.847294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:40.848223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:40.851472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 302/622] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:40.856756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:40.863172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-71b874e2-cad7-4357-853a-7426a56f5165 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:40.864486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-71b874e2-cad7-4357-853a-7426a56f5165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:40.886646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:40.942284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 321/623] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 27 08:20:40.943411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b874e2-cad7-4357-853a-7426a56f5165 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:40.982820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:40.999772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4c50b-5904-4e0a-87e8-9a98120ff422 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.045215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.047117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.048249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.048892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.050245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:41.050720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.053114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.077351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b874e2-cad7-4357-853a-7426a56f5165 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.078111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 303/624] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:20:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:41.079773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.119443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.120446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.120963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.121513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.122865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:41.123415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.125487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.151392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 322/625] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 27 08:20:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:41.151392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.164455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:41.195393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf2acf-5ea3-4ae8-97bd-4fa863506260 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.233976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.234892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.235393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.235882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.247060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:41.252736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:41.253834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:41.254895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:41.261425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f26cb5-81c5-4d15-ac91-649b2109103b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.261835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 304/626] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:41.262891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.267084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:41.296751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c1442-93a0-4dbb-979d-50915c06a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.297810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c1442-93a0-4dbb-979d-50915c06a7bd None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.297810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c1442-93a0-4dbb-979d-50915c06a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.298305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c1442-93a0-4dbb-979d-50915c06a7bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.370343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655f31b-cc3c-4a6e-80c7-1d305ecb5576 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.373808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c1442-93a0-4dbb-979d-50915c06a7bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.375317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c1442-93a0-4dbb-979d-50915c06a7bd None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.390732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:41.408963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.409632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.409885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.410175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.420348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.421792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:41.439487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:41.440393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:41.441392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:41.441392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:41.441392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:41.441392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:41.446167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 305/627] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:41.446516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.488096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 323/628] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:41.489289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeebc3-1566-45e3-8dae-6c73974e671a None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.489570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeebc3-1566-45e3-8dae-6c73974e671a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.489901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaeebc3-1566-45e3-8dae-6c73974e671a None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.491373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 324/629] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:41.491874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaeebc3-1566-45e3-8dae-6c73974e671a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.512583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:41.562884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168ea6a-da7f-4e85-abeb-919f6665836c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.601445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.602167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.602520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.602863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.603568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:41.603872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.605217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.607657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaeebc3-1566-45e3-8dae-6c73974e671a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.618436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 306/630] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:41.618436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.627261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:41.648378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b9b7-c51e-42a0-bfaa-3f6f5d50226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.648995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b9b7-c51e-42a0-bfaa-3f6f5d50226c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.649277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b9b7-c51e-42a0-bfaa-3f6f5d50226c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.649550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b9b7-c51e-42a0-bfaa-3f6f5d50226c None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.650272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b9b7-c51e-42a0-bfaa-3f6f5d50226c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:41.650582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b9b7-c51e-42a0-bfaa-3f6f5d50226c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.651968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b9b7-c51e-42a0-bfaa-3f6f5d50226c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.668478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 325/631] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:41.668478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc388ea-58cc-416c-a5e2-1f6410c73be7 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.668478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc388ea-58cc-416c-a5e2-1f6410c73be7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.668478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc388ea-58cc-416c-a5e2-1f6410c73be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.686270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2fc388ea-58cc-416c-a5e2-1f6410c73be7 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:41.687513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2fc388ea-58cc-416c-a5e2-1f6410c73be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:41.730570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccbe35-a8fc-4fb2-8bab-5e8d7bd5030e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.749431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:41.761409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 326/632] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:41.769722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.770442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.770728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.770998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.775014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc388ea-58cc-416c-a5e2-1f6410c73be7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.783334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.784964 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:41.800228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:41.801144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:41.802074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:41.802074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:41.802074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:41.802074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:41.802074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 307/633] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:41.802403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.888711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc388ea-58cc-416c-a5e2-1f6410c73be7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.892461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:41.912864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1939-b835-4090-bd78-c39d738f37e1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:41.925201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.925871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.926131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.926432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.936592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.938059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.948999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:41.949635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:41.949917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:41.950219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:41.953685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:41.955461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:41.956332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:41.956332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:41.956332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:41.956332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:41.956332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 327/634] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:41.957215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:41.960397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:41.962914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:41.976728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:41.977682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:41.978631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 308/635] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:41.979130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.009060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.060798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc8379-20cf-4881-a411-c4c731caadd9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.079458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350789e-9d0a-4c18-a087-77839c12d7bf None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.094207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.094916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.095202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.095732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.105489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:42.110925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:42.111934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:42.112763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 328/636] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:42.113832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a14f9-36ba-4450-b664-f6263595fda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.113922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.114000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a14f9-36ba-4450-b664-f6263595fda4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.114000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a14f9-36ba-4450-b664-f6263595fda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.114000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a14f9-36ba-4450-b664-f6263595fda4 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.114291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a14f9-36ba-4450-b664-f6263595fda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:42.114489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a14f9-36ba-4450-b664-f6263595fda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.115840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a14f9-36ba-4450-b664-f6263595fda4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.132854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 309/637] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:42.132854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd6275-8b5e-4b59-9da5-5f78abc51ba4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.132854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd6275-8b5e-4b59-9da5-5f78abc51ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.132854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd6275-8b5e-4b59-9da5-5f78abc51ba4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.132854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 310/638] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:42.132854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd6275-8b5e-4b59-9da5-5f78abc51ba4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.139864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.219507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.220867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1f0ce-bfb9-4f99-afac-7d8eba0a9715 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.232269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd6275-8b5e-4b59-9da5-5f78abc51ba4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.254057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.254698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.254979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.255253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.256000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:42.256309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.257675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.266285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.267359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.267737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.268011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.268750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:42.269058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.270484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.270719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 329/639] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:42.271503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.282076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 311/640] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:42.283095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.351816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.374484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380a9a0-9dbb-4cde-aac7-f557848f54b9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.383244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d2e48-1fe5-4eff-9065-07e8d2376e96 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.410240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4486c14-dcfc-4db9-8871-0649552ce029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.410861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486c14-dcfc-4db9-8871-0649552ce029 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.411133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486c14-dcfc-4db9-8871-0649552ce029 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.411397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486c14-dcfc-4db9-8871-0649552ce029 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.412099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4486c14-dcfc-4db9-8871-0649552ce029 None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:42.412401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4486c14-dcfc-4db9-8871-0649552ce029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.413766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4486c14-dcfc-4db9-8871-0649552ce029 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.417358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.417987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.418301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.418583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.421745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 330/641] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:42.423500 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc64775-3bfa-4473-9976-3b3f22ad92c2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.423921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc64775-3bfa-4473-9976-3b3f22ad92c2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.424254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc64775-3bfa-4473-9976-3b3f22ad92c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.426543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.427975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:42.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:42.441225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:42.442135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 312/642] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:42.442702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-dcc64775-3bfa-4473-9976-3b3f22ad92c2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:42.442778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.443022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dcc64775-3bfa-4473-9976-3b3f22ad92c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:42.456367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.510344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 331/643] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:42.511223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc64775-3bfa-4473-9976-3b3f22ad92c2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.549142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56679147-1a54-468c-a0e9-a9ae709dced1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.583741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.584405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.584688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.584952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.585828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.592827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.594320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:42.605968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:42.607146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 313/644] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:42.608185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.623483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc64775-3bfa-4473-9976-3b3f22ad92c2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.656668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.657977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.658301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.658576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/92173e230c3e45c3b769221fb4b3d666/roles/b2f748f19ea54f688475e569b0847642` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.672395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=92173e230c3e45c3b769221fb4b3d666, role_id=b2f748f19ea54f688475e569b0847642)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.674815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.675025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.707820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c40b16-1a30-455e-900f-9235e63c8c2b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.740417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.741037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.741313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.741577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.746000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 332/645] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:20:42 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/92173e230c3e45c3b769221fb4b3d666/roles/b2f748f19ea54f688475e569b0847642 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:42.746864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.753224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.754746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:42.771650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:42.772803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:42.773712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:42.773712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:42.773712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:42.773712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:42.773712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 314/646] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:42.774189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.864186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeabed-4261-456f-8ccf-3589bb9b7f6f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.883485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.888363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e455e-ca44-4654-9536-0f3c2b52eb75 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:42.906743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.907390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.907670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.907929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.908807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:42.909109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:42.911148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:42.916206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:42.923846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 333/647] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:42.924791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:42.936972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:42.937601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:42.937882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:42.938805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:42.949691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:42.955933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:42.961881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 315/648] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:42.962869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.040589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c056b19-5a14-46ca-9ba0-f4c9410230e5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.067870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f8aed-45a8-41e3-9d8c-3d21801c7218 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.076501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.077265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.077645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.078030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.079168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:43.079573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.081090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.100062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 334/649] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:43.101073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.108474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.109089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.109370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.109641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.110890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:43.111304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.112674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.130193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 316/650] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:43.130193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.144031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:43.174759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:43.214876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b99296-fd8b-4eaf-93c2-505ea1912636 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.248235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed7b1-f699-411b-b8dc-6417ceb99818 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.253761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.254410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.254694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.254966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.255754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e4e461b0b6142b38fd7de56395733b2) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:43.256066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.257584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.274182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 335/651] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/role_assignments?user.id=4e4e461b0b6142b38fd7de56395733b2&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:43.274182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.306198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.307241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.307592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.308395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.309477 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:43.309905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.312382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.325400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 317/652] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:43.326410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.392125 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:43.395891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e59d57-7e0d-4973-8ec4-7e2cc6a60ed6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.440702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.441425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.441780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.442128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.458308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11163e13-d66d-4218-86ef-993d7af89953 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.460106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:43.500216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.501658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.502031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.502456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.511035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.512598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:43.529218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:43.530398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:20:43.531560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:43.534222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 318/653] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:43.534222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.542162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.543633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.678874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c94db0-136b-40b1-982b-a1770c07e4fc None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.679066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:43.679774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 336/654] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:43.682914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.726878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.727530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.727813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.728090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.743202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.745932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:43.767913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:43.769275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:43.770272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:43.770272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:43.770272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:43.770272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:43.770272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 319/655] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:43.770272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.790381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:43.828346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1ae2f-38e5-4d79-9d33-cd2880f6f3e9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.873685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.875131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.875708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.876254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.877336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:43.878176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.881173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.906454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe453515-d2fc-4faf-b6c8-665dac3a7e95 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:43.910654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 337/656] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:43.911495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:43.958409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:43.959413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:43.959893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:43.960406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:43.975254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:43.977303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:43.991527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:44.001142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:44.002056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:44.003388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:44.003388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:44.003388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:44.003388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:44.006642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 320/657] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:44.009077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.051234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ba0d6-9678-42c6-929b-e949d6235633 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.090575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.090575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.090575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.090575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.090998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:44.091076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.092436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.106095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 338/658] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:44.111232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.115802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.165824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbbd29-e335-480e-a61d-3a729c063424 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.210807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.211752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.212228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.212717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.220823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29bd92-1242-4f1e-9020-47360db4a56e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.240479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.259683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.259683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.260047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.260475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.261622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:44.261940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.263358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.282817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 339/659] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:44.284060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.306199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.308044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.380334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.410572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db6656-5c4e-4c3a-8b0b-d70f182fdf11 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.426342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 321/660] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:44.427683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.470832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.471400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.471684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.471960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.479598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.481070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:44.497556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:44.498671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 27 08:20:44.500147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:44.504237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 340/661] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:44.505128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.544148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09663a-ffda-4360-910c-8adae53e7b71 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.561060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.595766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.596972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.597459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.597936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.605516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.607292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:44.619463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:44.620459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:44.621374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:44.623857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 322/662] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:44.625006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.656102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06b7f-7e54-4a83-8ee1-4f666984cd27 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.665415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.699481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.700883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.701741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.702464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.717584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.721312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:44.740512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:44.741688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:44.742719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:44.742719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:44.742719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:44.742719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:44.744888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 341/663] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:44.746280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.764716 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c541-39e6-4d89-8c5b-482b5630878d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.810738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.811421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.811613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.811875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.812663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:44.812996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.814673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.843159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 323/664] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:44.844049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.865067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.876033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec46ef9-e221-41c5-83f7-229784196e1f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:44.914713 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:44.915331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:44.915613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:44.915887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:44.916631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:44.916937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:44.918421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:44.935942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 342/665] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:44.935942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:44.947211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:44.964189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff1d76-0c1f-4f53-bdd0-004e076341c0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.002649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b15c2f-0fef-4fe4-88e2-f59ad1bf8db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.003186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15c2f-0fef-4fe4-88e2-f59ad1bf8db9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.003467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15c2f-0fef-4fe4-88e2-f59ad1bf8db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.003735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15c2f-0fef-4fe4-88e2-f59ad1bf8db9 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.004419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b15c2f-0fef-4fe4-88e2-f59ad1bf8db9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:45.004715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b15c2f-0fef-4fe4-88e2-f59ad1bf8db9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.006055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b15c2f-0fef-4fe4-88e2-f59ad1bf8db9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.022776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 324/666] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:45.022776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ecf5-8efa-41e3-ab91-2d6983006efa None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.022776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ecf5-8efa-41e3-ab91-2d6983006efa None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.022776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ecf5-8efa-41e3-ab91-2d6983006efa None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.025937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 325/667] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:45.026743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761ecf5-8efa-41e3-ab91-2d6983006efa None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.044488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eaa38-bd29-4fa0-8744-c6125dc4642b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.082554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bedf3-0efe-4c53-9937-f729768d1441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.083178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bedf3-0efe-4c53-9937-f729768d1441 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.083875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bedf3-0efe-4c53-9937-f729768d1441 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.084155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bedf3-0efe-4c53-9937-f729768d1441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.105966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:45.134610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761ecf5-8efa-41e3-ab91-2d6983006efa None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.157436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bedf3-0efe-4c53-9937-f729768d1441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.159186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bedf3-0efe-4c53-9937-f729768d1441 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.171769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.173070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.173540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.173978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.174948 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e41212825ad4990958a7e909c669177) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:45.175254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.176670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.182465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 326/668] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/role_assignments?user.id=4e41212825ad4990958a7e909c669177&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:45.183235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.203296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:45.286712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 343/669] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:45.288267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd503-80ba-46cb-a785-efd893e6c743 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.288724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd503-80ba-46cb-a785-efd893e6c743 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.289148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd503-80ba-46cb-a785-efd893e6c743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.310689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-facbd503-80ba-46cb-a785-efd893e6c743 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:45.312283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-facbd503-80ba-46cb-a785-efd893e6c743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:45.329574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517087-f6ab-4a68-834c-1efd449b9571 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.374287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.374931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.375205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.375543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.376986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:45.383745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.385321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:45.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:45.400291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:45.401391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:45.401906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 327/670] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:45.402220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.412747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 344/671] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:45.414041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd503-80ba-46cb-a785-efd893e6c743 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.492109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:45.530726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433aa098-3ca1-41b2-ac64-649b23f9ace0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.532476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd503-80ba-46cb-a785-efd893e6c743 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.567827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3c92c-729c-45be-8342-48e1eb6da2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.569478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3c92c-729c-45be-8342-48e1eb6da2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.569994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3c92c-729c-45be-8342-48e1eb6da2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.570564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3c92c-729c-45be-8342-48e1eb6da2d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.575263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.575886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.576168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.576442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.577195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:45.577504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.579952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.582652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3c92c-729c-45be-8342-48e1eb6da2d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.584271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db3c92c-729c-45be-8342-48e1eb6da2d6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.591794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 345/672] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:45.592406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9db3c92c-729c-45be-8342-48e1eb6da2d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:45.603439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:45.604357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:45.605350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:45.605350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:45.605350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:45.605350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:45.611570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 328/673] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:45.613009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3357fab-d713-4d3d-a7e3-46bcc607dc7a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.613437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3357fab-d713-4d3d-a7e3-46bcc607dc7a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.613865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3357fab-d713-4d3d-a7e3-46bcc607dc7a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.615150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 329/674] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:45.615830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3357fab-d713-4d3d-a7e3-46bcc607dc7a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.644511 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:45.705855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3ad16-cf9c-4da3-b609-295b97d0a1e5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.728616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3357fab-d713-4d3d-a7e3-46bcc607dc7a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.743160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.743920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.744649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.744649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.751904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:45.755412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:45.760683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:45.761828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 346/675] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:45.764598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.771686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59958e27-f357-41aa-a97c-06c1ba5fd39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.772307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958e27-f357-41aa-a97c-06c1ba5fd39d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.772593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958e27-f357-41aa-a97c-06c1ba5fd39d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.772871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958e27-f357-41aa-a97c-06c1ba5fd39d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.773679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59958e27-f357-41aa-a97c-06c1ba5fd39d None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=99a29a23f8a84076bab3adefc042a088) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:45.773975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59958e27-f357-41aa-a97c-06c1ba5fd39d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.775461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59958e27-f357-41aa-a97c-06c1ba5fd39d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.781557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 330/676] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/role_assignments?user.id=99a29a23f8a84076bab3adefc042a088&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:45.783671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcb4f-d902-43e2-b4e7-4701969c6b4f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.784357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcb4f-d902-43e2-b4e7-4701969c6b4f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.785011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcb4f-d902-43e2-b4e7-4701969c6b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.805998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-540bcb4f-d902-43e2-b4e7-4701969c6b4f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:45.807173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-540bcb4f-d902-43e2-b4e7-4701969c6b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:45.888124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 331/677] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:45.889275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bcb4f-d902-43e2-b4e7-4701969c6b4f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:45.919238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:45.924995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac02dc8-8488-4d4c-a946-87929c8cd7b2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:45.973763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94769315-4483-416b-a44e-d0da45218bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:45.973763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94769315-4483-416b-a44e-d0da45218bea None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:45.973763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94769315-4483-416b-a44e-d0da45218bea None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:45.973763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94769315-4483-416b-a44e-d0da45218bea None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:45.974441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94769315-4483-416b-a44e-d0da45218bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:45.974747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94769315-4483-416b-a44e-d0da45218bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:45.976127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94769315-4483-416b-a44e-d0da45218bea None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:45.992656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 347/678] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:45.993223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94769315-4483-416b-a44e-d0da45218bea None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.019026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bcb4f-d902-43e2-b4e7-4701969c6b4f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.040682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:46.063710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.064325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.064600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.064868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.065632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:46.065938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.067385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.080984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 332/679] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:46.081977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.121147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94769315-4483-416b-a44e-d0da45218bea None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.164011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.164666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.164948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.165220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.176956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:46.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:46.183138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:46.183969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 348/680] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:46.184655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.207277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21a66-a14a-4178-92f6-15f72560732f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.211980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:46.255022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.256528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.257086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.257615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.267731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.269376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:46.285830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:46.286927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:46.288044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:46.290838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 333/681] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:46.291789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.304931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9ddb-80e1-49c5-b7e8-94006e2d2c57 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.329282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:46.351420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.352630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.353063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.353786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.368239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.370853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:46.389963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:46.391612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:46.392493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:46.392493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:46.392493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:46.392493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:46.394286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 349/682] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:46.395280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.407895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed3af1-99d3-4b81-9f36-7553b83b5628 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.444291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.444939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.445281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.446056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.446595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:46.446967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.448389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.463286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 334/683] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:46.463286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.484580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:46.516634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87cc22-3031-4aa7-9e1f-c3c9d0cefdda None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.555714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.556696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.557198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.557680 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.558664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:46.559201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.560842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.570898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:46.575348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08c7e-9ee2-474d-a1f7-39f933a2063d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.581530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 350/684] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:46.584845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.618646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.619216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.619496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.619764 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.621161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:46.621161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.622182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.640534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 335/685] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:46.641819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.690518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77580291-d233-4405-baf2-c61e867ebd8f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.721489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:46.726550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.727297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.727681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.728056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.738506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.740037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.751601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aad98-9142-4723-a9be-8283290c7dd4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:46.756612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:46.757704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:46.758617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:46.758617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:46.758617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:46.758617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:46.763636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 351/686] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:46.764845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.793149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.794210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.794767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.795248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.796234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=d5c308c33c674f1ea7111c0297174c2f) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:46.796750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.799252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.814308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 336/687] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/role_assignments?user.id=d5c308c33c674f1ea7111c0297174c2f&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:46.815765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:46.868341 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:46.880106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b44c7-f8f1-4f3c-8b56-5bcaf88b5694 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.921702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:46.926566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:46.926882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:46.927162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:46.943707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:46.945186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:46.958636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0815dd-cff3-4606-b7d5-bb31a9717b80 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:46.964098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:46.965006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:46.965915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:46.970947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 352/688] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:46.973599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.002137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:47.003213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.003703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.004176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.005165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:47.005679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:47.007524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:47.022879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 337/689] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:47.035387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.058225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:47.117950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d2853-d890-429c-b18d-7286e94411e3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:47.135953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:47.153369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:47.154081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.155302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.156342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.157923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:47.158352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:47.159941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:47.160769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94369e96-3c30-404e-a4c5-17cfd2b86a62 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:47.178424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 353/690] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 27 08:20:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:47.183740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.201303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:47.202360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.202957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.204491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.212737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:47.214606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:47.227872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:47.228805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:47.229813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:47.232561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 338/691] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:20:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:47.243424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.300989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0b948-d041-43a4-85c3-fe1f3c4e4ff2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:47.302550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:47.337799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bf218-9bce-4080-aa2e-994fcafc9bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:47.338445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bf218-9bce-4080-aa2e-994fcafc9bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.338723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bf218-9bce-4080-aa2e-994fcafc9bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.338997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bf218-9bce-4080-aa2e-994fcafc9bf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.358057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a594d-a300-4acc-b05d-bc7be634f8ca None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:47.396793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:47.397945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.398476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.399000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.400132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:47.400651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:47.402388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:47.411007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bf218-9bce-4080-aa2e-994fcafc9bf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:47.412468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bf218-9bce-4080-aa2e-994fcafc9bf0 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:47.419048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:47.424199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 339/692] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:20:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:47.430937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.543232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 354/693] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 27 08:20:47.544457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8a8d8-3bec-4558-8bc3-fd2386d358ba None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.544740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8a8d8-3bec-4558-8bc3-fd2386d358ba None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.545012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8a8d8-3bec-4558-8bc3-fd2386d358ba None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.546524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 355/694] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:47.551945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8a8d8-3bec-4558-8bc3-fd2386d358ba None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.556239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:47.567784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a3a6d-9e69-41d9-87cc-e6e6dcc92ca2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:47.607517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93eba79-4a27-4e82-96ab-6744f73ebe92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:47.608392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eba79-4a27-4e82-96ab-6744f73ebe92 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.608849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eba79-4a27-4e82-96ab-6744f73ebe92 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.609319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eba79-4a27-4e82-96ab-6744f73ebe92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.669040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8a8d8-3bec-4558-8bc3-fd2386d358ba None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:47.692190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:47.695085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93eba79-4a27-4e82-96ab-6744f73ebe92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:47.696585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93eba79-4a27-4e82-96ab-6744f73ebe92 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:47.815083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 340/695] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 27 08:20:47.816503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2154fa-6386-486b-affd-9ca7b5093e9b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.816885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2154fa-6386-486b-affd-9ca7b5093e9b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.817255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2154fa-6386-486b-affd-9ca7b5093e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.834615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:47.835519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:47.835944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:47.836358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:47.836807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-da2154fa-6386-486b-affd-9ca7b5093e9b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:47.838291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-da2154fa-6386-486b-affd-9ca7b5093e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:47.848623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:47.850397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:47.866599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:47.867557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:47.868503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:47.868503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:47.868503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:47.868503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:47.874386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 356/696] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:47.875512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.918166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 341/697] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:47.918166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2154fa-6386-486b-affd-9ca7b5093e9b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:47.950915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:47.958301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.009877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229437e-71d9-4bd6-aa70-64d19fa295fd None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.019405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2154fa-6386-486b-affd-9ca7b5093e9b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.049778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f34707-4c7d-4911-9526-29b6a75e8f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.050605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f34707-4c7d-4911-9526-29b6a75e8f8b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.050986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f34707-4c7d-4911-9526-29b6a75e8f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.051386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f34707-4c7d-4911-9526-29b6a75e8f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.057467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.058081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.058495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.058762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.059444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:48.059751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.061098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.062899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f34707-4c7d-4911-9526-29b6a75e8f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.064411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f34707-4c7d-4911-9526-29b6a75e8f8b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.074616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 342/698] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:48.074616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-b8f34707-4c7d-4911-9526-29b6a75e8f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:48.081709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:48.082847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:48.083950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:48.083950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:48.083950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:48.083950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:48.090706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 357/699] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:48.090706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23267f0-bab6-4e76-ba83-f4a683229194 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.090706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23267f0-bab6-4e76-ba83-f4a683229194 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.090706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23267f0-bab6-4e76-ba83-f4a683229194 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.090706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 358/700] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:48.090706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23267f0-bab6-4e76-ba83-f4a683229194 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.181677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.190846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a2281-d1dc-490b-9beb-f228f23a6a8e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.205147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23267f0-bab6-4e76-ba83-f4a683229194 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.213092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.227140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.227902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.228306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.228784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.241106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.242658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.243279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c05a0-9a45-4621-87f5-f55f8a535d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.243947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c05a0-9a45-4621-87f5-f55f8a535d79 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.244236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c05a0-9a45-4621-87f5-f55f8a535d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.244754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c05a0-9a45-4621-87f5-f55f8a535d79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.252150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c05a0-9a45-4621-87f5-f55f8a535d79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.253579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c05a0-9a45-4621-87f5-f55f8a535d79 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:48.258526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:48.259424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:48.260311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:48.260311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:48.260311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:48.260311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:48.260311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 343/701] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:48.260594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-f82c05a0-9a45-4621-87f5-f55f8a535d79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:48.270445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:48.271439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:48.272333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 359/702] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:48.273263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7eb7c-f042-4917-88da-236a05b05d2c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.273361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7eb7c-f042-4917-88da-236a05b05d2c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.273627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7eb7c-f042-4917-88da-236a05b05d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.291818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-78e7eb7c-f042-4917-88da-236a05b05d2c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:48.292978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-78e7eb7c-f042-4917-88da-236a05b05d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:48.374243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 360/703] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:48.374243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7eb7c-f042-4917-88da-236a05b05d2c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.375177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345b862-e728-487c-bd3d-e7826ef35248 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.415220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.416354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.416354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.416581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.417252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:48.417563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.419434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.433536 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 344/704] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:48.434262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.437157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.481734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.487965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7eb7c-f042-4917-88da-236a05b05d2c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.522268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.522882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.523157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.523424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.524110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:48.524408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.525760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.536828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 361/705] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:48.537629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.546102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce23b0-3fe1-4b0b-b8ed-2298e8f159df None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.594082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.594728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.595008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.595534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.596278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:48.596583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.597998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.610181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 345/706] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:48.610838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.652978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343969f1-0af0-4071-9d95-6195adc8eaed None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.677304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.689897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.695114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.695114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.695114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.695114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.695114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:48.705678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:48.706842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:20:48.707938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:48.709164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 362/707] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:48.709771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.724458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58974f-4682-44fb-809b-41ec576f4903 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.735307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.768232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.769094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.769423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.769693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.778573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.779996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:48.791458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:48.792509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 346/708] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:48.798417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.827880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a2a3b-8826-4ecf-aa52-0800c46feada None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.868018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.868912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.869188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.869459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.870131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.871837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.901933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e886e-2ba1-4879-b98f-8c5f730bd379 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:48.924046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:48.935149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:48.935787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:48.936065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:48.936335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:48.937120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=92173e230c3e45c3b769221fb4b3d666) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:48.937431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:48.939579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:48.945303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 347/709] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/role_assignments?user.id=92173e230c3e45c3b769221fb4b3d666&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:48.946081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:48.964512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 363/710] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:20:48 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:48.964512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:49.143633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:49.158390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d4fea-8f97-4344-9c12-77a35cfd9a88 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:49.186067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc97be-8124-474a-8167-6ff053d1ec0b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:49.206506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:49.207135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.207415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.207685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.208543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:49.208850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:49.210247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:49.216384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 348/711] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:49.223333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:49.235762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:49.236415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.236685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.236958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.237723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:49.238031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:49.240012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:49.259175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 364/712] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:49.268207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:49.293859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:49.355860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794db72e-3826-4e1a-8355-d81ba3f21735 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:49.398834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35548729-5c84-4415-89eb-958c9953d71e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:49.399772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cbf9a-bfa6-4afa-948a-8290f8efa9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:49.400653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cbf9a-bfa6-4afa-948a-8290f8efa9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.400934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cbf9a-bfa6-4afa-948a-8290f8efa9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.402105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cbf9a-bfa6-4afa-948a-8290f8efa9c6 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.402923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cbf9a-bfa6-4afa-948a-8290f8efa9c6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:49.403237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cbf9a-bfa6-4afa-948a-8290f8efa9c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:49.404599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cbf9a-bfa6-4afa-948a-8290f8efa9c6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:49.412734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 349/713] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 27 08:20:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:49.414499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76701fc-9dad-46b9-80e9-1cf222ce57af None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.414787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76701fc-9dad-46b9-80e9-1cf222ce57af None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.415061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76701fc-9dad-46b9-80e9-1cf222ce57af None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.418291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 350/714] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:49.423952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba77749-a161-44ea-a890-ef050b6bc981 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.424235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba77749-a161-44ea-a890-ef050b6bc981 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.424508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba77749-a161-44ea-a890-ef050b6bc981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.447484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bba77749-a161-44ea-a890-ef050b6bc981 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:49.448786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bba77749-a161-44ea-a890-ef050b6bc981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:49.450441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:49.451327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.451606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.451925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.454354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e4e461b0b6142b38fd7de56395733b2) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:49.454674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:49.456195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:49.460345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:49.466492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 365/715] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:49 2022] GET /identity/v3/role_assignments?user.id=4e4e461b0b6142b38fd7de56395733b2&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:49.560925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 351/716] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:49.609596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:49.730049 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a5d34-2a4a-4fd0-87ef-71eb46197f66 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:49.767833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:49.768488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.768817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.769098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] PATH_INFO: `/v3/services/network` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.769731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:49.771135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:49.780610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:49.781636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:49.782651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 366/717] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:49 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:49.789705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba77749-a161-44ea-a890-ef050b6bc981 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:49.864800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:49.937041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba77749-a161-44ea-a890-ef050b6bc981 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:49.983963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6fa11-f0b9-45b8-af67-f2d647a26698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:49.985011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6fa11-f0b9-45b8-af67-f2d647a26698 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:49.985431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6fa11-f0b9-45b8-af67-f2d647a26698 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:49.985804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6fa11-f0b9-45b8-af67-f2d647a26698 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:49.986820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6fa11-f0b9-45b8-af67-f2d647a26698 None admin] RBAC: Adding query filter params (name=network) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:49.987247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6fa11-f0b9-45b8-af67-f2d647a26698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:49.988956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6fa11-f0b9-45b8-af67-f2d647a26698 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:49.994338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 352/718] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:20:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:50.002524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:50.147134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:50.197831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55004a10-0983-4b7c-88a7-62a0803210fb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:50.248292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cd991-9f23-4282-b2b6-3b63298f1511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:50.248840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cd991-9f23-4282-b2b6-3b63298f1511 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:50.249119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cd991-9f23-4282-b2b6-3b63298f1511 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:50.249387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cd991-9f23-4282-b2b6-3b63298f1511 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:50.250072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cd991-9f23-4282-b2b6-3b63298f1511 None admin] RBAC: Adding query filter params (type=network) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:50.251045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cd991-9f23-4282-b2b6-3b63298f1511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:50.252429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cd991-9f23-4282-b2b6-3b63298f1511 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:50.263135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 367/719] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:20:50 2022] GET /identity/v3/services?type=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:51.593482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869744c-148f-42e2-975a-6bbc8d3cc821 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:51.594187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869744c-148f-42e2-975a-6bbc8d3cc821 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:51.594187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869744c-148f-42e2-975a-6bbc8d3cc821 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:51.596024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 353/720] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:51.601261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5b82c-4270-47f6-9f70-5f69083df210 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:51.601686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5b82c-4270-47f6-9f70-5f69083df210 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:51.601806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5b82c-4270-47f6-9f70-5f69083df210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:51.629642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-19f5b82c-4270-47f6-9f70-5f69083df210 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:51.630866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-19f5b82c-4270-47f6-9f70-5f69083df210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:51.722857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 368/721] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:51.758904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869744c-148f-42e2-975a-6bbc8d3cc821 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:51.794949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67223172-8f29-4211-978e-d0c43c9fd230 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:51.795304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67223172-8f29-4211-978e-d0c43c9fd230 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:51.795501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67223172-8f29-4211-978e-d0c43c9fd230 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:51.797228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 369/722] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:51.802818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced02e3-1e2b-49b3-8f24-f04baf46e5a9 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:51.803464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced02e3-1e2b-49b3-8f24-f04baf46e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:51.803936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced02e3-1e2b-49b3-8f24-f04baf46e5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:51.823815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5ced02e3-1e2b-49b3-8f24-f04baf46e5a9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:51.825459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5ced02e3-1e2b-49b3-8f24-f04baf46e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:51.885950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869744c-148f-42e2-975a-6bbc8d3cc821 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:51.897008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:51.897829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:51.913892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 370/723] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:51.915281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58788a9-88d7-4dbf-87c8-0d04478d5124 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:51.915567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58788a9-88d7-4dbf-87c8-0d04478d5124 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:51.915838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58788a9-88d7-4dbf-87c8-0d04478d5124 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:51.921066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 371/724] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:51.923022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f551-c115-4d55-96cb-5bcdeca9f242 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:51.923213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f551-c115-4d55-96cb-5bcdeca9f242 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:51.923482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f551-c115-4d55-96cb-5bcdeca9f242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:51.929298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:51.930391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:51.931666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:51.932162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:51.942445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-96a1f551-c115-4d55-96cb-5bcdeca9f242 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:51.943687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-96a1f551-c115-4d55-96cb-5bcdeca9f242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:51.946079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:51.952454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:51.954055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:51.954932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:51.956412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 354/725] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:51.961421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.029839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 372/726] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:52.031046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f551-c115-4d55-96cb-5bcdeca9f242 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.102473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087c1da-34ca-486b-94dc-e62f4dccba59 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.144509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.145161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.145440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.145709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.159542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:52.166592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:52.167602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:52.168442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 355/727] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:52.169133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.193456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f551-c115-4d55-96cb-5bcdeca9f242 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.203844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.215136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.232935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.233801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.234083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.234414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.235142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:52.235445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.237601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:52.246097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 373/728] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:52.246937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.297402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add60fe-5dcf-4413-b905-d86c027a90f9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.338901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d9200-fd7c-408e-b5b7-8e08dc74b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.339534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d9200-fd7c-408e-b5b7-8e08dc74b36b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.339808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d9200-fd7c-408e-b5b7-8e08dc74b36b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.340072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d9200-fd7c-408e-b5b7-8e08dc74b36b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.350423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efbe39-d7cf-447c-a2aa-a97baea268c4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.351608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d9200-fd7c-408e-b5b7-8e08dc74b36b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-de1d9200-fd7c-408e-b5b7-8e08dc74b36b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:52.356887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:52.357842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:52.358737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 356/729] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:52.359534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46164c-08b0-413a-8d14-59292fde2e99 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.360528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46164c-08b0-413a-8d14-59292fde2e99 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.360528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46164c-08b0-413a-8d14-59292fde2e99 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.361401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 357/730] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:52.362026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46164c-08b0-413a-8d14-59292fde2e99 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.397788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.398500 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.398720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.399033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.399696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:52.400000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.401358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:52.414908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 374/731] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:52.414908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.460241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.462282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.475942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46164c-08b0-413a-8d14-59292fde2e99 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.510607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7657a88-651b-4046-b2b5-baab141dda26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.511242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657a88-651b-4046-b2b5-baab141dda26 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.511542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657a88-651b-4046-b2b5-baab141dda26 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.511789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7657a88-651b-4046-b2b5-baab141dda26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.521574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21b013-2b57-465c-8ed0-89b038136b3d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.522888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7657a88-651b-4046-b2b5-baab141dda26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.524350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7657a88-651b-4046-b2b5-baab141dda26 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a7657a88-651b-4046-b2b5-baab141dda26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:52.540626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:52.541544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:52.543068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:52.543068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:52.543068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:52.543068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:52.543068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 358/732] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:52.543357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3519bd9-a629-4ae7-a976-d8bdb0f4999f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.543568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3519bd9-a629-4ae7-a976-d8bdb0f4999f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.543970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3519bd9-a629-4ae7-a976-d8bdb0f4999f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.566087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a3519bd9-a629-4ae7-a976-d8bdb0f4999f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:52.567291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a3519bd9-a629-4ae7-a976-d8bdb0f4999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:52.569420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.570797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.571085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.571355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.572074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:52.572377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.573734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:52.583571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 375/733] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:52.588514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.637244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 359/734] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:52.638010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3519bd9-a629-4ae7-a976-d8bdb0f4999f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.695985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eecfe9-6b2c-4e2b-8bf3-82159b94612b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.698446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.720559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.730843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.731479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.731761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.732036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.744451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.745898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:52.752586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3519bd9-a629-4ae7-a976-d8bdb0f4999f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:52.759885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:52.760904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:52.761809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:52.761809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:52.761809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:52.761809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:52.761809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 376/735] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:52.761809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.790751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.791353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.791625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.791901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.792571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:52.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.794232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:52.805776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 360/736] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:52.806643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.863471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4b67-4197-4264-b5dd-02b6a9919d00 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.896172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aefe74-b354-4b47-bcac-cb14d94e02e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.896721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefe74-b354-4b47-bcac-cb14d94e02e5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.896997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefe74-b354-4b47-bcac-cb14d94e02e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.897272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aefe74-b354-4b47-bcac-cb14d94e02e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.903983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1e172-8353-4ce4-ab93-7d073c4e8e3e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:52.909343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aefe74-b354-4b47-bcac-cb14d94e02e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.910814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aefe74-b354-4b47-bcac-cb14d94e02e5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-f9aefe74-b354-4b47-bcac-cb14d94e02e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:52.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:52.925766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:52.927005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:52.927005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:52.927005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:52.927005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:52.927005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 377/737] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:52.927304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a161f-b04c-40eb-b104-60f53d0714c3 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.927419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a161f-b04c-40eb-b104-60f53d0714c3 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.927684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a161f-b04c-40eb-b104-60f53d0714c3 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.928906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 378/738] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:52.929589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a161f-b04c-40eb-b104-60f53d0714c3 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:52.938082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.941168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:52.943854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:52.944536 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:52.950555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:52.950555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:52.950555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:52.950555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.027764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a161f-b04c-40eb-b104-60f53d0714c3 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.050883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 361/739] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:20:52 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:53.053638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.068056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.068689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.068967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.069259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.069972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:53.070359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.071706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.085280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 379/740] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:53.087548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.171697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:53.214895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fb42a-ea11-4529-9bca-f688469812f4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.215095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5767151-ea99-4558-a60f-a9a4ba83ddfd None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.255788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.256021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f0e23-c3b9-485f-9fad-5fdf4e6c9232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.256414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f0e23-c3b9-485f-9fad-5fdf4e6c9232 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.256526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.256697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.256792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f0e23-c3b9-485f-9fad-5fdf4e6c9232 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.256950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f0e23-c3b9-485f-9fad-5fdf4e6c9232 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.257041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.257653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f0e23-c3b9-485f-9fad-5fdf4e6c9232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:53.257930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f0e23-c3b9-485f-9fad-5fdf4e6c9232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.259333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f0e23-c3b9-485f-9fad-5fdf4e6c9232 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.264731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.266724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.271151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 380/741] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:53.272211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809585ad-e3a9-485a-8e70-8f805182c791 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.272497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809585ad-e3a9-485a-8e70-8f805182c791 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.272771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809585ad-e3a9-485a-8e70-8f805182c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:53.283974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:53.284941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 362/742] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:53.290540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.297734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-809585ad-e3a9-485a-8e70-8f805182c791 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:53.298960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-809585ad-e3a9-485a-8e70-8f805182c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:53.345494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:53.386189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 381/743] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 27 08:20:53.386189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809585ad-e3a9-485a-8e70-8f805182c791 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.409889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71b53b-9327-4dca-9052-504e91a22961 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.446529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:53.449816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.450813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.451096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.451643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.462496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.463985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:53.475787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:53.476766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 363/744] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:53.477771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.500729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809585ad-e3a9-485a-8e70-8f805182c791 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.538551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.539092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.539374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.539649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.547811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.549269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:53.561622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:53.562743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 382/745] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:53.563820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.596717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a1113-9757-414b-b36f-8dd72539f26d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.607667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:53.635112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.635729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.636010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.636281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.637066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:53.637367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.638835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.662179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 364/746] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:53.662179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.678991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf82be-d5a2-42eb-9747-ef487b653ea3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.694533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:53.715199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.715819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.716014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.716287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.716981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.718514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.762554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5a90d-ce25-409a-baaf-2c3199693a43 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.782173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 383/747] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:20:53 2022] POST /identity/v3/services => generated 240 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:20:53.782173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.833385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.834347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.834700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.835424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.836313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:53.836689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.839739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.867324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 365/748] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:53.868397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:53.907832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:53.913155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8979f-86d3-4f9a-af27-dc7c413348f6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:53.959972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:53.960588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:53.960861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:53.961126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:53.961914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:53.962890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:53.963654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:53.984506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 384/749] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:53.985692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.006105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d903e-34a5-44ff-a4b6-8e6646594419 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.029232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.050652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.052526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.052526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.052526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.122456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b5e6-bdf1-4e7b-999e-376ff7807237 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.141140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.142916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.160982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.161851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.162321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.162766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.169092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.238886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.240587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.251809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 366/750] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:20:54.252807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.273412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.352612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 385/751] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:20:54.353850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.358749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d605918-c515-446e-84dd-085d95b4ad9b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.396279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.402235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.403082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.403272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.403517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.415054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.416545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:54.432330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:54.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:54.434177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:54.434177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:54.434177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:54.434177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:54.434177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 367/752] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:54.434459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.462467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fefe5-52c9-4184-8823-d8ca1313640e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.494268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.500559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.501616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.501936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.506355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.513335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.514910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:54.530995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:54.531982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:54.532860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:54.532860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:54.532860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:54.532860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:54.537095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 386/753] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:54.538351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.546356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b80c6-abe7-425d-8798-8ef39a3a9137 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.584367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.585430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.585891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.586385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.598828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.600732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:54.621342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:54.622411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:54.623524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:54.623524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:54.623524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:54.623524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:54.626116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 368/754] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:54.627101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.656763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcaa01-f557-4f72-913a-a5c2a9e6a723 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.674602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.695123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.695858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.700470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.700470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.700470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:54.700470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.700470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.713855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 387/755] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:54.715011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.763120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc19cd4-6982-48eb-a0ed-32e0abbc4f5a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.764673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.805138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.806115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.806465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.806702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.807430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:54.807732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.809718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.824838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 369/756] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:54.825883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.834429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8ece-be6c-4533-b6bc-0cc1e4d1728f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.878417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.879764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.880537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.881043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.882124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:54.883074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.885128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:54.900813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 388/757] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:54.902193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:54.927693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:54.947296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4345a-dcc8-4c63-931a-6dbfee1e938c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:54.988966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:54.989601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:54.989874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:54.990160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:54.997435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:54.999197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:55.011295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:55.016165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:55.017071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:55.017956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 370/758] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:55.018661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.029377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8f9f7-a8b4-4dcc-8004-0f306bbe9b97 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.034684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:55.071036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.071977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.072466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.072917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.081351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.083296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:55.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:55.097949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:55.099101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:55.100612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 389/759] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:55.105398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.145507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7679c6-57b1-419f-a46d-e44c820b4b85 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.183192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.184048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.184332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.184669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.192858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.194384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.204876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab3c8e-1955-4ae4-987b-71a9729438db None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.242761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728e430-84e7-4dc4-a602-e044c0d119fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.243392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e430-84e7-4dc4-a602-e044c0d119fe None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.243665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e430-84e7-4dc4-a602-e044c0d119fe None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.243932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728e430-84e7-4dc4-a602-e044c0d119fe None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.244694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728e430-84e7-4dc4-a602-e044c0d119fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:55.244999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728e430-84e7-4dc4-a602-e044c0d119fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.246518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728e430-84e7-4dc4-a602-e044c0d119fe None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.264880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 390/760] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:55.264880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472f64-db31-4f56-91b5-030f2a14cfb2 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.264880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472f64-db31-4f56-91b5-030f2a14cfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.264880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472f64-db31-4f56-91b5-030f2a14cfb2 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.264880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 391/761] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:55.264880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01472f64-db31-4f56-91b5-030f2a14cfb2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.293748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:20:55.333636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:55.334651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:20:55.335524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:55.340473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 371/762] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:55.341222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.350990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:55.376845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01472f64-db31-4f56-91b5-030f2a14cfb2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.413301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181692f-7c74-4b84-856d-46e4430c6af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.414031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181692f-7c74-4b84-856d-46e4430c6af9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.414311 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181692f-7c74-4b84-856d-46e4430c6af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.414539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181692f-7c74-4b84-856d-46e4430c6af9 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.415320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181692f-7c74-4b84-856d-46e4430c6af9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:55.415619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181692f-7c74-4b84-856d-46e4430c6af9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.417098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181692f-7c74-4b84-856d-46e4430c6af9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.429872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 392/763] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:55.431166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846320b-d635-476f-a5cb-9fbbdec36f4e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.431453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846320b-d635-476f-a5cb-9fbbdec36f4e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.431727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846320b-d635-476f-a5cb-9fbbdec36f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.453527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-0846320b-d635-476f-a5cb-9fbbdec36f4e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:55.454840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0846320b-d635-476f-a5cb-9fbbdec36f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:55.455208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee37c-5101-4993-b516-c7e0994ef512 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.489355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.490171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.490473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.490738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e4e461b0b6142b38fd7de56395733b2/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.492655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=4e4e461b0b6142b38fd7de56395733b2, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.494068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.524365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 393/764] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:20:55.534326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846320b-d635-476f-a5cb-9fbbdec36f4e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.551537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:55.594548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 372/765] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:20:55 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e4e461b0b6142b38fd7de56395733b2/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:20:55.594548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.658666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846320b-d635-476f-a5cb-9fbbdec36f4e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.684834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:55.699744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133da800-790c-4db8-9273-084e2f62049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.700764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133da800-790c-4db8-9273-084e2f62049d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.701048 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133da800-790c-4db8-9273-084e2f62049d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.701383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133da800-790c-4db8-9273-084e2f62049d None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.702734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133da800-790c-4db8-9273-084e2f62049d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:55.703205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133da800-790c-4db8-9273-084e2f62049d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.704982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133da800-790c-4db8-9273-084e2f62049d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.723333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 394/766] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:55.724321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133da800-790c-4db8-9273-084e2f62049d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.735837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cccc6c-7357-4fa4-960f-1508a0a05b66 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.781842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.782635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.782913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.783184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.783972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=92173e230c3e45c3b769221fb4b3d666) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:55.784281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.785707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.792516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 373/767] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/role_assignments?user.id=92173e230c3e45c3b769221fb4b3d666&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:55.793148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.844822 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:55.880131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133da800-790c-4db8-9273-084e2f62049d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:55.938553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:55.939370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:55.939555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:55.939836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:55.940477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:55.941879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:55.951803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:55.952811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:55.954415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 395/768] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:55.959269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:55.976452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f95bc-b100-489e-aa8d-5a48da3a9657 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.013172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:56.020004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42d4ec-ec6c-4ab5-aa48-c72e70033a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.020651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42d4ec-ec6c-4ab5-aa48-c72e70033a14 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.020925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42d4ec-ec6c-4ab5-aa48-c72e70033a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.021192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42d4ec-ec6c-4ab5-aa48-c72e70033a14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.022038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42d4ec-ec6c-4ab5-aa48-c72e70033a14 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=03d7fd886b4843efa9c022741bb85542) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:56.022397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42d4ec-ec6c-4ab5-aa48-c72e70033a14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.023819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42d4ec-ec6c-4ab5-aa48-c72e70033a14 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.029844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 374/769] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/role_assignments?user.id=03d7fd886b4843efa9c022741bb85542&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:56.031890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cc318-5dd4-435c-a345-755a3d41bc36 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.032193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cc318-5dd4-435c-a345-755a3d41bc36 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.032463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cc318-5dd4-435c-a345-755a3d41bc36 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.042617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 375/770] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:56.051651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c1bb8-2d03-4ee5-9d5d-4c23b9aa652a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.051959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c1bb8-2d03-4ee5-9d5d-4c23b9aa652a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.052215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c1bb8-2d03-4ee5-9d5d-4c23b9aa652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.099314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-013c1bb8-2d03-4ee5-9d5d-4c23b9aa652a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:56.100435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-013c1bb8-2d03-4ee5-9d5d-4c23b9aa652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:56.102323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c12528-89f7-4950-9488-2a4a58ccb8d0 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.144856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.145497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.145781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.146069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.146782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:56.147087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.148508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.161646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 396/771] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 27 08:20:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:56.167420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:56.171243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:56.236134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 376/772] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:20:56.259827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c1bb8-2d03-4ee5-9d5d-4c23b9aa652a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:56.308875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30339e-e988-4313-b361-b4b6dd909e89 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.350201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.350766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.351042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.351313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.351990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:56.352290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.357442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.370869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 397/773] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 27 08:20:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:56.379143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:56.417066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c1bb8-2d03-4ee5-9d5d-4c23b9aa652a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.443151 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:56.462989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.463644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.463961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.464241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] PATH_INFO: `/v3/services/network` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.464928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.466315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:56.485163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:56.486244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:56.487083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 377/774] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:20:56 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:56.499316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:56.518544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:56.552450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e719f49-89d9-454e-a6dc-f54f3d2b9b9d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.594681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.595924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.596410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.596910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.597867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] RBAC: Adding query filter params (interface=public, service_id=20244c14e9df4ae48cb23d942f482528, region_id=RegionOne) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:56.598405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.600416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.610809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 398/775] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:20:56 2022] GET /identity/v3/endpoints?service_id=20244c14e9df4ae48cb23d942f482528&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:56.619170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:56.714229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467c3b6-f640-42cd-8060-89d3594f1194 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.764511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.766227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.766839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.767316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.768200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] RBAC: Adding query filter params (name=network) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:56.768753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.770405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.775646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 378/776] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:20:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:56.782044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:56.836926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b807a3a-78a3-471b-9021-6bceea348f58 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.838550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:56.886691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0debc9-7e1f-4a5a-a75b-482f74717bba None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:56.924541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:56.926631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.927359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.927760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.928136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.928928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] RBAC: Adding query filter params (type=network) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:56.929333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.930986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.936742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 379/777] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:20:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:56.942938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:56.944625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:56.945263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:56.945531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:56.945801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:56.946478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:56.947861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:56.967011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 399/778] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:20:56 2022] GET /identity/v3/services => generated 772 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:57.053507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58ab63-f979-498b-ae08-3e1ace8a187e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:57.071365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:57.091684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64546d-5129-4ec4-aab7-64e702c77a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:57.092306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64546d-5129-4ec4-aab7-64e702c77a8c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:57.092589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64546d-5129-4ec4-aab7-64e702c77a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:57.092856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64546d-5129-4ec4-aab7-64e702c77a8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:57.094332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64546d-5129-4ec4-aab7-64e702c77a8c None admin] RBAC: Adding query filter params (interface=public, service_id=b09d45de324646669f494fe8040f02fc, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:57.094764 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64546d-5129-4ec4-aab7-64e702c77a8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:57.096390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64546d-5129-4ec4-aab7-64e702c77a8c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:57.104818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 380/779] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:20:56 2022] GET /identity/v3/endpoints?service_id=b09d45de324646669f494fe8040f02fc&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:20:57.111159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:57.221044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db771bb6-03c6-4cc2-ba27-edc1014cf883 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:57.257328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5208f11-7d86-4a13-91b7-f94fabf098b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:57.257950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5208f11-7d86-4a13-91b7-f94fabf098b4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:57.258258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5208f11-7d86-4a13-91b7-f94fabf098b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:57.258531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5208f11-7d86-4a13-91b7-f94fabf098b4 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:57.259190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5208f11-7d86-4a13-91b7-f94fabf098b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:57.260576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5208f11-7d86-4a13-91b7-f94fabf098b4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:57.266760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 400/780] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:20:57 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:57.976040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2349e8-f8be-43df-afc9-975d18eea905 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:57.976558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2349e8-f8be-43df-afc9-975d18eea905 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:57.976558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2349e8-f8be-43df-afc9-975d18eea905 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:57.978049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 381/781] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:57.983336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f0525-a4e0-48a5-ba9e-6f797d37ee00 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:57.984174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f0525-a4e0-48a5-ba9e-6f797d37ee00 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:57.984653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f0525-a4e0-48a5-ba9e-6f797d37ee00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.003724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a24f0525-a4e0-48a5-ba9e-6f797d37ee00 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:58.005224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a24f0525-a4e0-48a5-ba9e-6f797d37ee00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:58.060293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:58.074682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 401/782] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:20:58.098169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2349e8-f8be-43df-afc9-975d18eea905 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:58.215504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2349e8-f8be-43df-afc9-975d18eea905 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:58.225102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e798447-dad6-47d2-8a14-d7da1e71adc9 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.225387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e798447-dad6-47d2-8a14-d7da1e71adc9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.225648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e798447-dad6-47d2-8a14-d7da1e71adc9 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.227349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 402/783] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:58.236794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26cfb-f16c-4a96-91d7-0bb2081f00cb None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.237074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26cfb-f16c-4a96-91d7-0bb2081f00cb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.237350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26cfb-f16c-4a96-91d7-0bb2081f00cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.260668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:58.261458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.261737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.262003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.271194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d0a26cfb-f16c-4a96-91d7-0bb2081f00cb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:58.272405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d0a26cfb-f16c-4a96-91d7-0bb2081f00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:58.274025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:58.279879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:58.280795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:58.282379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:58.288551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 382/784] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:58.299497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:58.384993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:58.408313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 403/785] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:20:58.435144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a26cfb-f16c-4a96-91d7-0bb2081f00cb None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:58.465866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592f473-7960-4fc8-a0a8-530762e73f85 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:58.508981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:58.509702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.509984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.510288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.511009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:58.511316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:58.512683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:58.535875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 383/786] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:58.541358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:58.645385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a26cfb-f16c-4a96-91d7-0bb2081f00cb None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:58.667816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb31a-40ae-45a6-809a-3a1d81b9bbb1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:58.669659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:58.682601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:58.692507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22864770-3255-4380-b512-b508bcd71a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:58.693938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22864770-3255-4380-b512-b508bcd71a70 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.694470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22864770-3255-4380-b512-b508bcd71a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.695256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22864770-3255-4380-b512-b508bcd71a70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.708729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae3a5f-41aa-4bd9-ab9c-0267aa3685cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:58.709413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae3a5f-41aa-4bd9-ab9c-0267aa3685cd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.709682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae3a5f-41aa-4bd9-ab9c-0267aa3685cd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.709951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae3a5f-41aa-4bd9-ab9c-0267aa3685cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.710930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22864770-3255-4380-b512-b508bcd71a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-22864770-3255-4380-b512-b508bcd71a70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:58.717172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:58.718122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:20:58.719153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:58.726341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 404/787] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:58.726341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c9ca-22b9-421d-964f-b53ae053420f None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.726341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c9ca-22b9-421d-964f-b53ae053420f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.726341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c9ca-22b9-421d-964f-b53ae053420f None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.726341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 405/788] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:58.726341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0c9ca-22b9-421d-964f-b53ae053420f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:58.729755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae3a5f-41aa-4bd9-ab9c-0267aa3685cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:58.731260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae3a5f-41aa-4bd9-ab9c-0267aa3685cd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e9ae3a5f-41aa-4bd9-ab9c-0267aa3685cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:58.748683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:58.750184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:58.751426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:58.751426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:58.751426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:58.751426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:58.753642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 384/789] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:20:58.755170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3304-7635-493c-b227-7f833143281a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.755678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3304-7635-493c-b227-7f833143281a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.756155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be3304-7635-493c-b227-7f833143281a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.775178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-11be3304-7635-493c-b227-7f833143281a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:58.776660 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-11be3304-7635-493c-b227-7f833143281a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:20:58.858388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 385/790] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:20:58.860343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be3304-7635-493c-b227-7f833143281a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:58.869747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0c9ca-22b9-421d-964f-b53ae053420f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:58.927292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:58.927961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:58.928247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:58.928519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:58.929524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:58.929832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:58.931558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:58.946255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 406/791] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:20:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:58.946255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:58.975838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:58.993151 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be3304-7635-493c-b227-7f833143281a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.004849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:59.033808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.034281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.034666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.035101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.035845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:59.036583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.038059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.051606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 386/792] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:59.052623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.084506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2123e44-0c31-4241-bb2b-0a3cb77e69bf None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.121833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.122653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.122880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.123162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.134723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:20:59.140842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:59.141845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:59.142717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 407/793] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:20:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:59.143431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.188313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc44081-c5fa-44f1-b94c-7f51ed8bacf7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.225370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.226017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.226345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.226618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.240382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.242220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.246154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:59.264480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:59.266027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:59.267085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:59.267085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:59.267085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:59.267085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:59.268411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 387/794] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:59.269439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.278789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:59.281987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fffa4a-1c45-42a6-8f5f-841e4678d1c1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.328493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.329769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.330543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.330829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.342680 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.344176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:59.356627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:59.357622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 408/795] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:59.358617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.399580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560800c8-03f4-4dc9-a2ca-1e28732445b7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.435020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.435636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.435913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.436188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.436907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:59.438013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.439404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.452917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 388/796] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:59.453988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.486651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfe0ae-54ad-4b6d-ab70-eaad67c81779 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.523960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:59.530289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.531537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.531878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.532157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.532859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:59.533164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.534750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.555560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 409/797] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:59.555560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.578325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965beeca-c995-4fe3-9961-750838195e06 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.579477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:59.619543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.620160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.620462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.620740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.621523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:59.621828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.623281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.645813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 389/798] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:59.645813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.668139 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a897aaa-e2ec-45e4-98ef-dbeb5cbafbc4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.700650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312932a-8210-4cb1-b124-2e898b692d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.701274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312932a-8210-4cb1-b124-2e898b692d90 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.701550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312932a-8210-4cb1-b124-2e898b692d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.701818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312932a-8210-4cb1-b124-2e898b692d90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.712119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312932a-8210-4cb1-b124-2e898b692d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.713560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312932a-8210-4cb1-b124-2e898b692d90 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-7312932a-8210-4cb1-b124-2e898b692d90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:20:59.729740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:59.730826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:59.731926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:20:59.731926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:20:59.731926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:20:59.731926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:20:59.731926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 410/799] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:59.732249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f3bb-6ad0-4446-b5d6-77a28c3ba249 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.732332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f3bb-6ad0-4446-b5d6-77a28c3ba249 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.732545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f3bb-6ad0-4446-b5d6-77a28c3ba249 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.733910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 411/800] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:59.737872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1f3bb-6ad0-4446-b5d6-77a28c3ba249 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.743057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9d00a-cc28-4331-96b6-aea532861f3c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.759572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:59.777075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccee960-86be-4e7d-9d0f-3a5cd4541ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.777789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccee960-86be-4e7d-9d0f-3a5cd4541ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.778060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccee960-86be-4e7d-9d0f-3a5cd4541ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.778353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccee960-86be-4e7d-9d0f-3a5cd4541ef0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.786580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccee960-86be-4e7d-9d0f-3a5cd4541ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.788005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccee960-86be-4e7d-9d0f-3a5cd4541ef0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7ccee960-86be-4e7d-9d0f-3a5cd4541ef0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:20:59.800194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:20:59.801302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:20:59.802300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 390/801] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:20:59.802881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741141ec-f585-4af3-9205-1dc474033fe6 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.802955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741141ec-f585-4af3-9205-1dc474033fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.803181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741141ec-f585-4af3-9205-1dc474033fe6 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.804407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 391/802] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:20:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:20:59.805093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741141ec-f585-4af3-9205-1dc474033fe6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:20:59.814200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:20:59.846682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1f3bb-6ad0-4446-b5d6-77a28c3ba249 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.879208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbef3b-8410-4ec3-b27a-a0ddc91acd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.879810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbef3b-8410-4ec3-b27a-a0ddc91acd4c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.880085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbef3b-8410-4ec3-b27a-a0ddc91acd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.880349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbef3b-8410-4ec3-b27a-a0ddc91acd4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.908944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741141ec-f585-4af3-9205-1dc474033fe6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:20:59.942652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029577-bd81-4e27-aca5-4b07c974b906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:20:59.943265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029577-bd81-4e27-aca5-4b07c974b906 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.943547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029577-bd81-4e27-aca5-4b07c974b906 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.943819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029577-bd81-4e27-aca5-4b07c974b906 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.944504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029577-bd81-4e27-aca5-4b07c974b906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:20:59.944817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029577-bd81-4e27-aca5-4b07c974b906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.946199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029577-bd81-4e27-aca5-4b07c974b906 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.947971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbef3b-8410-4ec3-b27a-a0ddc91acd4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:20:59.949411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbef3b-8410-4ec3-b27a-a0ddc91acd4c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:20:59.958445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 392/803] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:20:59.959745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1126f29-191b-4e4b-bc0d-856ea405c3f1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:20:59.960040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1126f29-191b-4e4b-bc0d-856ea405c3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:20:59.960315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1126f29-191b-4e4b-bc0d-856ea405c3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:20:59.977945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c1126f29-191b-4e4b-bc0d-856ea405c3f1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:20:59.979185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c1126f29-191b-4e4b-bc0d-856ea405c3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:00.008506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.019435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.044969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 393/804] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:20:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:00.045796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1126f29-191b-4e4b-bc0d-856ea405c3f1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.048841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 412/805] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:20:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:00.050071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3bdef-f120-4e7a-ab90-cacb4f3d0e16 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.050378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3bdef-f120-4e7a-ab90-cacb4f3d0e16 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.050651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3bdef-f120-4e7a-ab90-cacb4f3d0e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.068455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-78b3bdef-f120-4e7a-ab90-cacb4f3d0e16 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:00.069713 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-78b3bdef-f120-4e7a-ab90-cacb4f3d0e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:00.136233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 413/806] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:00.136996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3bdef-f120-4e7a-ab90-cacb4f3d0e16 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.165506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1126f29-191b-4e4b-bc0d-856ea405c3f1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.199542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.200155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.200433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.200705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.201484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:00.201787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.203273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.220619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 394/807] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:00.221425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.240854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3bdef-f120-4e7a-ab90-cacb4f3d0e16 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.242375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.245084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.274355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.274985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.275287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.275559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.283156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.284582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:00.295935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:00.297383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:00.298421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:00.302299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 414/808] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:00.303322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.318808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c25cd-769b-417d-9084-1dfd16e7bc4c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.351785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.352405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.352681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.352947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.363514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.364956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:00.379627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:00.380637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:00.381519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:00.381519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:00.381519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:00.381519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:00.381519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 395/809] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:00.381801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.404509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561239d6-e635-478b-8301-f3d9d4b78eb5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.439050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.439687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.439969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.440245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.440872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.442281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:00.450945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:00.452018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:00.458083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 415/810] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:00.459278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.461586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.484373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e974b-58ef-4812-84b2-f5bf6034bcbe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.484557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.518417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.519149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.519504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.519779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] PATH_INFO: `/v3/services/network` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.520408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.521765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:00.530476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:00.531798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:00.532828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 396/811] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/services/network => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:00.533782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.561675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76b7d1-148f-42f9-9940-20f73ad18fb1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.598281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.598912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.599186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.599462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.610197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.611630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:00.626469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:00.627542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:00.628728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:00.628728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:00.628728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:00.628728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:00.628728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 416/812] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:00.628728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.639720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269e53f-b30a-46e5-b38b-8930f64ef246 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.673276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.673808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.674083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.674381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.675170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:00.675468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.676873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.686101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.694057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 397/813] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:00.694907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.706077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.727689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc9a2c-98ec-431d-8ec2-947b91c8311d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.760725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.761341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.761619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.761891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.762604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:00.762916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.764274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.780078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 417/814] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:00.780078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.793275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09208725-08b6-4898-8ccf-dc82256b672a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.826973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.827678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.827960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.828234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.828898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:00.829384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.830572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.834913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 398/815] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:00.835678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.877904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d32916-bc83-412c-bc2d-b74ac6bbd221 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.899630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.910893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.911507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.911787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.913068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.913068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:00.913278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.914519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.918960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 418/816] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:00.919727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:00.922897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:00.938442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adff59-0aa3-4a9f-8968-ebad0ce2eb19 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:00.973693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:00.974331 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:00.974610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:00.974876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:00.975624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:00.975925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:00.977280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:00.988717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 399/817] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:00.989532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.021566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4559b-2d79-4cd5-978a-4aef6b0ddec6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.054641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.055271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.055545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.055812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.067550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.068975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:01.082754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:01.083765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:01.084652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:01.084652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:01.084652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:01.084652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:01.084652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 419/818] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:01.084944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.092133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b721a-6014-4a02-bbe2-f1d655241250 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.115454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.127798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.129930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.129930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.129930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.135736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.137176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.142390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:01.148620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:01.149598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:01.150579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 400/819] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:01.151062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.190656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4385-914d-46a7-b689-2b148813ffa4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.223503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.224115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.224392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.224667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.225335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:01.225636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.227024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.231653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 420/820] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:01.232416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.248713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b49538-533e-4aa5-a364-68cdd513b7e6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.282122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.282840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.283123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.283402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.284075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] RBAC: Adding query filter params (type=network) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:01.284381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.285732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.290763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 401/821] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:01.291369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.333482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08c99a-dffa-4069-9311-1586f7eac7f9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.339965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.350555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.367213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.367841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.368116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.368440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.375606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.377086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:01.389041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:01.390114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:01.391224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 421/822] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:01.391809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.400688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08650054-8ab8-43b5-93d7-282483b82307 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.434260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.434875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.435153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.435424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.436107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:01.436414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.438458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.449881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 402/823] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:01.450729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.497685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b626-1740-440e-82f8-bbde0a5065e8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.531882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.532493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.532772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.533054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.533797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:01.534101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.535555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.547274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa8e5b-f419-47f1-865b-657ee454a30b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.547412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 422/824] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:01.548084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.555670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.586572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.587180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.587456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.587725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.588423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.589781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.594321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.651853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd4318-9d95-4e14-9c93-69bf9775efe9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.686492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.687157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.687429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.687699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.688398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.689773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.696857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 403/825] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:01.697691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.771627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 423/826] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:01.772882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.842603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1949c-1218-42ef-8abb-94b39d304433 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.901922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.902587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.902866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.903246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.904005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:01.904310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.907411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:01.908166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.921311 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ffdd6-b92b-4604-bd5f-9ceeb8751287 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:01.929235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 404/827] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:01.930788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:01.947889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:01.976807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:01.977449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:01.977728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:01.977994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:01.987084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:01.988540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:02.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:02.002924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 424/828] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:02.004248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:02.077336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60fcbd-64f8-4e2c-bea5-d8b3319a6f48 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:02.114951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:02.115640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:02.116267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:02.116546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:02.117334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e4e461b0b6142b38fd7de56395733b2) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:02.117642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:02.119351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:02.128916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa42a6-dfe2-4a94-beb6-873eab96e6b6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:02.130442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 405/829] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:01 2022] GET /identity/v3/role_assignments?user.id=4e4e461b0b6142b38fd7de56395733b2&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:02.131195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:02.168735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:02.169427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:02.169709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:02.169989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/92173e230c3e45c3b769221fb4b3d666/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:02.172727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=92173e230c3e45c3b769221fb4b3d666, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:02.174248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:02.236694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:02.261583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c967f-37e6-4a26-b8d0-112aa945167c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:02.270460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 425/830] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:21:02 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/92173e230c3e45c3b769221fb4b3d666/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:21:02.308182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bd9b8-d8f4-4435-acac-bb39ac918c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:02.309175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd9b8-d8f4-4435-acac-bb39ac918c97 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:02.309702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd9b8-d8f4-4435-acac-bb39ac918c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:02.310211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bd9b8-d8f4-4435-acac-bb39ac918c97 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:02.312445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bd9b8-d8f4-4435-acac-bb39ac918c97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:02.313108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bd9b8-d8f4-4435-acac-bb39ac918c97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:02.314842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bd9b8-d8f4-4435-acac-bb39ac918c97 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:02.340713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 406/831] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:02.347653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:02.381735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:02.482464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883c55-2495-4d0d-8d7f-d218ec3c64ce None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:02.520069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53008053-c29a-46df-bb33-08884df02dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:02.520747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53008053-c29a-46df-bb33-08884df02dcd None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:02.521040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53008053-c29a-46df-bb33-08884df02dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:02.521304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53008053-c29a-46df-bb33-08884df02dcd None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/03d7fd886b4843efa9c022741bb85542/roles/b2f748f19ea54f688475e569b0847642` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:02.537109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53008053-c29a-46df-bb33-08884df02dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=03d7fd886b4843efa9c022741bb85542, role_id=b2f748f19ea54f688475e569b0847642)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:02.538834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53008053-c29a-46df-bb33-08884df02dcd None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:02.627332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 426/832] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:21:02 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/03d7fd886b4843efa9c022741bb85542/roles/b2f748f19ea54f688475e569b0847642 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:21:03.833011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e340-7177-499a-85f4-88b6d2f7dfc8 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:03.833527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e340-7177-499a-85f4-88b6d2f7dfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:03.833605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e340-7177-499a-85f4-88b6d2f7dfc8 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:03.835205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 407/833] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:03.840321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f8bf0-ec08-4c6f-93c7-68a6159b4297 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:03.840711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f8bf0-ec08-4c6f-93c7-68a6159b4297 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:03.840875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f8bf0-ec08-4c6f-93c7-68a6159b4297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:03.859253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4d2f8bf0-ec08-4c6f-93c7-68a6159b4297 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:03.860485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4d2f8bf0-ec08-4c6f-93c7-68a6159b4297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:03.919727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:03.928492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 427/834] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:03.950524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268e340-7177-499a-85f4-88b6d2f7dfc8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:04.059344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268e340-7177-499a-85f4-88b6d2f7dfc8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:04.096592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:04.097159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.097437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.097708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.098386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:04.099745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:04.109434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:04.110604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:04.111432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 408/835] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:04.116852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f8bf0-ec08-4c6f-93c7-68a6159b4297 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:04.224243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f8bf0-ec08-4c6f-93c7-68a6159b4297 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:04.267823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f7ece-46bf-48e5-89a9-2b9a448ff67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:04.268450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f7ece-46bf-48e5-89a9-2b9a448ff67b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.268732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f7ece-46bf-48e5-89a9-2b9a448ff67b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.269012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f7ece-46bf-48e5-89a9-2b9a448ff67b None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.269695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f7ece-46bf-48e5-89a9-2b9a448ff67b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:04.270002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f7ece-46bf-48e5-89a9-2b9a448ff67b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:04.271418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f7ece-46bf-48e5-89a9-2b9a448ff67b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:04.276693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 428/836] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:04.283452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:04.286310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:04.391844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48b2ae-1ccb-41a0-a428-80124e899679 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:04.396137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b542cb-11a3-4f0d-bb68-dc4bd65c3eaa None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.396423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b542cb-11a3-4f0d-bb68-dc4bd65c3eaa None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.396697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b542cb-11a3-4f0d-bb68-dc4bd65c3eaa None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.398180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 429/837] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:04.407795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220096c7-bc39-463c-80a6-a2dee4398236 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.408342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220096c7-bc39-463c-80a6-a2dee4398236 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.408814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220096c7-bc39-463c-80a6-a2dee4398236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.427308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-220096c7-bc39-463c-80a6-a2dee4398236 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:04.428819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-220096c7-bc39-463c-80a6-a2dee4398236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:04.435191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca29a7-74cc-41d6-b2f5-3dbe58ed8aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:04.435816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca29a7-74cc-41d6-b2f5-3dbe58ed8aba None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.436097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca29a7-74cc-41d6-b2f5-3dbe58ed8aba None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.436373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca29a7-74cc-41d6-b2f5-3dbe58ed8aba None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.437826 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca29a7-74cc-41d6-b2f5-3dbe58ed8aba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:04.438161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca29a7-74cc-41d6-b2f5-3dbe58ed8aba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:04.439614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca29a7-74cc-41d6-b2f5-3dbe58ed8aba None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:04.450179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 409/838] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:04 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:04.489862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9865f-51b7-4482-a455-ceb3240f9476 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.490289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9865f-51b7-4482-a455-ceb3240f9476 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.490573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9865f-51b7-4482-a455-ceb3240f9476 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.492103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 410/839] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:04.498961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0068c-3dc6-41ed-82c5-b0b07cb38ce9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.499926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0068c-3dc6-41ed-82c5-b0b07cb38ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.500192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:04.500384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0068c-3dc6-41ed-82c5-b0b07cb38ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.518430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 430/840] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:04.521870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-68c0068c-3dc6-41ed-82c5-b0b07cb38ce9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:04.526461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-68c0068c-3dc6-41ed-82c5-b0b07cb38ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:04.541126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220096c7-bc39-463c-80a6-a2dee4398236 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:04.598164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:04.608208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 411/841] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:04.632406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0068c-3dc6-41ed-82c5-b0b07cb38ce9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:04.660525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220096c7-bc39-463c-80a6-a2dee4398236 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:04.700600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce45242a-6790-4ce7-9d26-edd74503509f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:04.701246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45242a-6790-4ce7-9d26-edd74503509f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.701525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45242a-6790-4ce7-9d26-edd74503509f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.701798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce45242a-6790-4ce7-9d26-edd74503509f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.703004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45242a-6790-4ce7-9d26-edd74503509f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:04.704424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce45242a-6790-4ce7-9d26-edd74503509f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-ce45242a-6790-4ce7-9d26-edd74503509f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:04.713462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:04.714657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:04.715691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:04.718452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 431/842] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 27 08:21:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:04.723479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a557cba-26b1-4015-af9d-f64eb139efed None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.723768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a557cba-26b1-4015-af9d-f64eb139efed None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.724042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a557cba-26b1-4015-af9d-f64eb139efed None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.725334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 432/843] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:04.729545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a557cba-26b1-4015-af9d-f64eb139efed None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:04.753213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0068c-3dc6-41ed-82c5-b0b07cb38ce9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:04.786771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:04.787666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355f70d-c7de-494b-95d5-6ff52988cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:04.788297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f70d-c7de-494b-95d5-6ff52988cb31 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.788566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f70d-c7de-494b-95d5-6ff52988cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.788843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355f70d-c7de-494b-95d5-6ff52988cb31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.799510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355f70d-c7de-494b-95d5-6ff52988cb31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8355f70d-c7de-494b-95d5-6ff52988cb31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:04.804338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:04.805308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:04.806127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 412/844] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:21:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:04.810219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021c6ad-c11e-42bc-8563-adbded530a72 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.810219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021c6ad-c11e-42bc-8563-adbded530a72 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.810219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021c6ad-c11e-42bc-8563-adbded530a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.826890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9021c6ad-c11e-42bc-8563-adbded530a72 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:04.828281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9021c6ad-c11e-42bc-8563-adbded530a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:04.840360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a557cba-26b1-4015-af9d-f64eb139efed None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:04.879607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:04.880466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:04.880747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:04.881019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:04.881693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:04.881993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:04.882864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:04.883384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:04.888247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 433/845] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:04.888848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:04.907898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 413/846] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:04.909031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021c6ad-c11e-42bc-8563-adbded530a72 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.021425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f7e4-127b-4ff8-8b13-7e34e18dcb83 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.027627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021c6ad-c11e-42bc-8563-adbded530a72 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.031788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.050038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.050802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.051149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.051495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.052278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:05.052651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.054096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.064392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.065011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.065292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.065563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.066268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:05.066573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.068130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.068599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 434/847] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:21:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:05.069624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.078611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 414/848] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:04 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:05.080662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.151521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.191447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343d014-2303-432c-afaf-4a741500a93b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.221198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2072ac-bb04-4799-861d-69479aad3b7a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.229948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.230708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.231088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.231367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.248705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:05.255494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:05.256460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:05.257289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 435/849] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:05.263318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.264018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.264372 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.264728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.265198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.277242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.278777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:05.294400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:05.295325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:05.296979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:05.296979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:05.296979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:05.296979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:05.296979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 415/850] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:05.301597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.317147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.374842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c157ce-7bf6-441c-b985-bd9a96f662e0 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.411225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.411932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.412282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.412624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.413409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:05.413790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.415511 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.429012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 436/851] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:05.429984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.430097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66904d46-91d2-4d2c-ae8a-ffda7132ec42 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.432620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.473475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.474107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.474419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.474694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.475390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:05.475695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.477069 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.494095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 416/852] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:05.495015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.546687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.552238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8a458-0e47-481f-a833-6e29dcb3e718 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.597684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.598354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.598636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.598912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.611406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.612868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.619131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf018f-5d57-4435-93d7-64ce9f8cf2a6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:05.627926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:05.628894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:05.629770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:05.629770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:05.629770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:05.629770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:05.629770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 437/853] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:05.639140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.656950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.657610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.657887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.658179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.666238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.667688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:05.679883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:05.680848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 27 08:21:05.681781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:05.685670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 417/854] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:05.687203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.716863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.773533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956fa12-8431-401c-9deb-f53c71cbd5ed None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.803396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1ea07-e1d7-4b07-8bc2-6258110ce354 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.812462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.813090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.813368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.813638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.814357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:05.814662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.816479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.837376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 438/855] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:05.839061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.843871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.844552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.844840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.845375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:05.846348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:05.846651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:05.848060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:05.859018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.869181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 418/856] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:05.873972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:05.957699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5beca-9196-42e5-8979-b3fc382440e2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.966927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:05.989693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbb0ee-9d5a-4efc-bf71-3caf04e6be2f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:05.995295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:05.995980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:05.996259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:05.996530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.004438 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.005891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:06.018286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:06.021397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:06.022349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 439/857] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:06.025678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.032876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.033724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.034206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.034664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.102682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.106594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.108277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.134581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38c4c6-9cf9-4c8b-ae2a-0fdc78cbf3db None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.172060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.172767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.172967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.173220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.174013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:06.174559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.176137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.199663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 440/858] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:06.200985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.201721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.236057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 419/859] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:06.243773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.317660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558edfd-fd04-4914-acd3-74cead46e6f3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.349024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.356176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.356728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.357005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.357611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.367171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227de815-c957-4ffa-a462-a701990eda74 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.403253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.404310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.404802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.405289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.416499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.418293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.432498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.433951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:06.434690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:06.435534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:06.436445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:06.436445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:06.436445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:06.436445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:06.443765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 420/860] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:06.444867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.477243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.548725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 441/861] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:06.549478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c96698-94b6-4183-9bd0-15f937748988 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.550358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.584750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.585956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0e93a-2571-4ec7-8fef-9c506fd8c19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.586441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0e93a-2571-4ec7-8fef-9c506fd8c19b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.586653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0e93a-2571-4ec7-8fef-9c506fd8c19b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.586952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0e93a-2571-4ec7-8fef-9c506fd8c19b None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.587608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0e93a-2571-4ec7-8fef-9c506fd8c19b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:06.587927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0e93a-2571-4ec7-8fef-9c506fd8c19b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.589282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0e93a-2571-4ec7-8fef-9c506fd8c19b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.601280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 421/862] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:06.602539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79991e35-a5be-4994-87e0-13dbfbc03c65 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.602822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79991e35-a5be-4994-87e0-13dbfbc03c65 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.603112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79991e35-a5be-4994-87e0-13dbfbc03c65 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.604506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 422/863] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:06.609892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79991e35-a5be-4994-87e0-13dbfbc03c65 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.669726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e35099-adeb-49e9-970d-98aa625c65b4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.704639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cb300-1b58-4709-8cf0-4b54f4f2e108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.705278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cb300-1b58-4709-8cf0-4b54f4f2e108 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.705555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cb300-1b58-4709-8cf0-4b54f4f2e108 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.705819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cb300-1b58-4709-8cf0-4b54f4f2e108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.707137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.716202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cb300-1b58-4709-8cf0-4b54f4f2e108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.717061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79991e35-a5be-4994-87e0-13dbfbc03c65 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.720375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cb300-1b58-4709-8cf0-4b54f4f2e108 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-786cb300-1b58-4709-8cf0-4b54f4f2e108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:06.734359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:06.736081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:06.737302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:06.737302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:06.737302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:06.737302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:06.738858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 442/864] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:06.740407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c6b8f-9196-4c39-b881-d542cd074bf0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.740687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c6b8f-9196-4c39-b881-d542cd074bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.740958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c6b8f-9196-4c39-b881-d542cd074bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.759638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-870c6b8f-9196-4c39-b881-d542cd074bf0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:06.759947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.760497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.760774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.760888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-870c6b8f-9196-4c39-b881-d542cd074bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:06.761060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.768301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.769854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:06.782169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:06.783068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:06.783944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 423/865] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:06.784419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.832257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 443/866] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:06.833437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c6b8f-9196-4c39-b881-d542cd074bf0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.855278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.902887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224a40b-d23f-41f7-82bc-cab4df46e7b5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.940529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.941140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.941416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.941684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.942487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:06.942688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.944041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:06.949360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c6b8f-9196-4c39-b881-d542cd074bf0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:06.956833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 424/867] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:06.957430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:06.973494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:06.992682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:06.993308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:06.993581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:06.993850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:06.994621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:06.994924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:06.996343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:07.013409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 444/868] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:07.013409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:07.066295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1da4da-9811-430e-b968-3036064e40ce None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:07.102063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:07.103029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb128d6a-28dc-4f78-9a39-cd6e2bc4a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:07.103525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128d6a-28dc-4f78-9a39-cd6e2bc4a42b None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.103807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128d6a-28dc-4f78-9a39-cd6e2bc4a42b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.104078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128d6a-28dc-4f78-9a39-cd6e2bc4a42b None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.104762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb128d6a-28dc-4f78-9a39-cd6e2bc4a42b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:07.106133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb128d6a-28dc-4f78-9a39-cd6e2bc4a42b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:07.132387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d25bd-ea07-473a-b4c8-32e43c3f04cc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:07.166382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:07.166948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.167229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.167499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.174575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:07.176032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:07.187669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:07.189318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 445/869] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:07.191031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:07.216807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 425/870] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:21:06 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:07.218792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a148cb-3d16-468f-88a0-6fae79cbaa79 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.219149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a148cb-3d16-468f-88a0-6fae79cbaa79 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.219630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a148cb-3d16-468f-88a0-6fae79cbaa79 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.221228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 426/871] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:07.226062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a148cb-3d16-468f-88a0-6fae79cbaa79 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:07.305439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c3cbc-c845-4597-9eb0-7e62cc398fc2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:07.306046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:07.341316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:07.345448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a148cb-3d16-468f-88a0-6fae79cbaa79 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:07.349970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478ec46-200a-428d-a166-d304b26878d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:07.350652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ec46-200a-428d-a166-d304b26878d2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.350933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ec46-200a-428d-a166-d304b26878d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.351240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478ec46-200a-428d-a166-d304b26878d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.352019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ec46-200a-428d-a166-d304b26878d2 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=92173e230c3e45c3b769221fb4b3d666) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:07.352331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ec46-200a-428d-a166-d304b26878d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:07.354386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478ec46-200a-428d-a166-d304b26878d2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:07.361871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 446/872] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:07 2022] GET /identity/v3/role_assignments?user.id=92173e230c3e45c3b769221fb4b3d666&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:07.363657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012961f-a716-49f4-92ae-ed800f71b747 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.364019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012961f-a716-49f4-92ae-ed800f71b747 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.364372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012961f-a716-49f4-92ae-ed800f71b747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.387359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f012961f-a716-49f4-92ae-ed800f71b747 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:07.388909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f012961f-a716-49f4-92ae-ed800f71b747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:07.409226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:07.409917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.410236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.410531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.411299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:07.411607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:07.413064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:07.431225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 427/873] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:07.438698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:07.492707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 447/874] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:07.515720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012961f-a716-49f4-92ae-ed800f71b747 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:07.572112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6c08f-6a3e-4a8a-8f55-8107248c50b5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:07.607709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:07.616072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9636fe0-8fdc-4143-b7ca-c288b532b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:07.616687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636fe0-8fdc-4143-b7ca-c288b532b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.621105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636fe0-8fdc-4143-b7ca-c288b532b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.621105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636fe0-8fdc-4143-b7ca-c288b532b5a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.621105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636fe0-8fdc-4143-b7ca-c288b532b5a7 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=03d7fd886b4843efa9c022741bb85542) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:07.621105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636fe0-8fdc-4143-b7ca-c288b532b5a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:07.621105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636fe0-8fdc-4143-b7ca-c288b532b5a7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:07.626418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 428/875] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:07 2022] GET /identity/v3/role_assignments?user.id=03d7fd886b4843efa9c022741bb85542&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:07.649696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012961f-a716-49f4-92ae-ed800f71b747 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:07.681890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:07.692669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73728cc8-e80d-45a3-81de-dd82cbdc50b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:07.693592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728cc8-e80d-45a3-81de-dd82cbdc50b6 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:07.694038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728cc8-e80d-45a3-81de-dd82cbdc50b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:07.694510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73728cc8-e80d-45a3-81de-dd82cbdc50b6 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:07.695385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73728cc8-e80d-45a3-81de-dd82cbdc50b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:07.696956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73728cc8-e80d-45a3-81de-dd82cbdc50b6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:07.765299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 448/876] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:21:07 2022] POST /identity/v3/services => generated 269 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:21:09.344366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e4c6a-14a8-4c74-b9c1-25b514e75bbd None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.345656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e4c6a-14a8-4c74-b9c1-25b514e75bbd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.346137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e4c6a-14a8-4c74-b9c1-25b514e75bbd None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.347921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 429/877] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:09.353486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cac92-26e4-4e90-a3b0-cb8258f4c242 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.354373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cac92-26e4-4e90-a3b0-cb8258f4c242 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.354850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cac92-26e4-4e90-a3b0-cb8258f4c242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.374647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e29cac92-26e4-4e90-a3b0-cb8258f4c242 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:09.376142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e29cac92-26e4-4e90-a3b0-cb8258f4c242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:09.456718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 449/878] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:09.479205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e4c6a-14a8-4c74-b9c1-25b514e75bbd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:09.581884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab500-2202-4172-badf-76285caa7b36 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.582510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab500-2202-4172-badf-76285caa7b36 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.583002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab500-2202-4172-badf-76285caa7b36 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.584803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 450/879] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:09.590097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8167-40b9-4105-983a-d7b9cc4258ff None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.590671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8167-40b9-4105-983a-d7b9cc4258ff None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.591141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c8167-40b9-4105-983a-d7b9cc4258ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.592545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e4c6a-14a8-4c74-b9c1-25b514e75bbd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:09.611632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-288c8167-40b9-4105-983a-d7b9cc4258ff None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:09.613264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-288c8167-40b9-4105-983a-d7b9cc4258ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:09.636767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:09.637685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.637972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.638275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.638904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:09.641261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:09.653832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:09.655027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:09.656601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 430/880] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:09.663651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:09.699969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 451/881] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:09.722339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c8167-40b9-4105-983a-d7b9cc4258ff None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:09.733979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:09.766595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:09.783254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25f2df-c473-4539-96db-4dc8a4e4bffe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:09.820712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210b9a8-2e59-4b46-a788-25023e83d891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:09.821324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210b9a8-2e59-4b46-a788-25023e83d891 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.821601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210b9a8-2e59-4b46-a788-25023e83d891 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.821879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210b9a8-2e59-4b46-a788-25023e83d891 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.822571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210b9a8-2e59-4b46-a788-25023e83d891 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:09.822871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210b9a8-2e59-4b46-a788-25023e83d891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:09.824216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210b9a8-2e59-4b46-a788-25023e83d891 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:09.826536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c8167-40b9-4105-983a-d7b9cc4258ff None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:09.829520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 431/882] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:09.830694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b6a7b-2629-49dd-bfca-fad4d1518043 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.830977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b6a7b-2629-49dd-bfca-fad4d1518043 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.831247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b6a7b-2629-49dd-bfca-fad4d1518043 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.832584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 432/883] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:09.833422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d57b-8a32-41d0-86e0-6d01086c4701 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.833757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d57b-8a32-41d0-86e0-6d01086c4701 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.834261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215d57b-8a32-41d0-86e0-6d01086c4701 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.836537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 433/884] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:09.836537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215d57b-8a32-41d0-86e0-6d01086c4701 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:09.863931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe1904-3413-45ae-a9fb-272eaa09b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:09.864492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1904-3413-45ae-a9fb-272eaa09b264 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.864772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1904-3413-45ae-a9fb-272eaa09b264 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.865041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1904-3413-45ae-a9fb-272eaa09b264 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.865667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe1904-3413-45ae-a9fb-272eaa09b264 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:09.867079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe1904-3413-45ae-a9fb-272eaa09b264 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-24fe1904-3413-45ae-a9fb-272eaa09b264 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:09.876537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:09.877709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:09.878592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 452/885] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:09.879396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c4c08-9ef6-45fc-b644-423987faef6b None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.879396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c4c08-9ef6-45fc-b644-423987faef6b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.879396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c4c08-9ef6-45fc-b644-423987faef6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.897120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-103c4c08-9ef6-45fc-b644-423987faef6b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:09.898352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-103c4c08-9ef6-45fc-b644-423987faef6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:09.938782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215d57b-8a32-41d0-86e0-6d01086c4701 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:09.958867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:09.965327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 453/886] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:09.966756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8c3c6-69b3-47f5-8bdc-fdc404408564 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.967054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8c3c6-69b3-47f5-8bdc-fdc404408564 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.970585 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8c3c6-69b3-47f5-8bdc-fdc404408564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.973062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:09.973670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:09.973946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:09.974244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:09.975141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:09.975260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:09.976533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:09.981405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 434/887] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:09.982026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:09.990224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3ac8c3c6-69b3-47f5-8bdc-fdc404408564 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:09.991414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3ac8c3c6-69b3-47f5-8bdc-fdc404408564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:10.021357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.058967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 454/888] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:10.059782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8c3c6-69b3-47f5-8bdc-fdc404408564 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.083912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69b773-eaec-462d-8032-c5d4ccd2642e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.118061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.118636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.118919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.119188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.119862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:10.120163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.121524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.126748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 435/889] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:10.127530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.161192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8c3c6-69b3-47f5-8bdc-fdc404408564 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.176834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.194612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.195231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.195509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.195782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.196531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] RBAC: Adding query filter params (interface=public, service_id=72e10b4af0784821ac35248b75c2e8f5, region_id=RegionOne) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:10.196842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.198241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.203700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 455/890] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/endpoints?service_id=72e10b4af0784821ac35248b75c2e8f5&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:10.204485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.231524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63af667-1635-4a98-bc33-0cd321ee3a8b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.235744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.265244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.265877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.266164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.266453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.276455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:10.282432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:10.283369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 436/891] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:10.284537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.309300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff82b5-8a4a-44d7-abe8-1b6cb089b2f2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.343661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.344263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.344547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.344812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.345706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.347102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:10.355802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:10.357295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:10.358788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 456/892] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:10.360073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.387226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e26693-233b-4f8c-a6fa-05ba600fa80e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.396688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.422210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.422865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.423414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.423692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.424371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:10.424683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.426046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.430466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 437/893] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:10.431275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.454397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.461698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace77e2-2bd5-4f39-b9e3-abff94f60be8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.502671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.503307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.503589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.503865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.504529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.506065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.522444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 457/894] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:21:10.522444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.589611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10357d8-3816-4b02-85e9-1d5a449450f5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.637900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.638901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.640142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.640460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.641308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:10.641621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.643874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.657776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 438/895] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:10.667223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.670364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15101c03-d952-41a9-9f95-876348138db5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.714316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.715238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.715356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.715559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.716566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.717340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:10.717694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.719122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.725453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 458/896] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:10.730528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.762413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.792838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681910-2578-4c15-946b-638ee950ebcc None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.835608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.836543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.836966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.837405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.848417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.850103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.864630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99f58d-8143-42ed-a41c-b7e1a97d2e71 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:10.866442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:10.867501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:10.868539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:10.868539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:10.868539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:10.868539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:10.870395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 439/897] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:10.876414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.918046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:10.918947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:10.919386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:10.919809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:10.922058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:10.922623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:10.924179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:10.930160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 459/898] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:10.935525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:10.978129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:10.997213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213bc4a-fd00-4c05-90bc-8840dcf3cdf7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:11.041950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:11.044469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:11.046188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:11.046619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:11.047091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:11.048190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:11.048867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:11.051080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:11.059360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6d530-fe0b-4104-bd76-429367957c32 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:11.067490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 440/899] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:11.072573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:11.103915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:11.105141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:11.106092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:11.106859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:11.107830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] RBAC: Adding query filter params (interface=public, service_id=4dfb91a9373647748aba6ebea2cc0c4b, region_id=RegionOne) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:11.109057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:11.111315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:11.118357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 460/900] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:21:10 2022] GET /identity/v3/endpoints?service_id=4dfb91a9373647748aba6ebea2cc0c4b&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:11.123760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:11.208055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebc23e-d8ac-4052-8beb-a64c23709eb7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:11.238972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2826d77-9242-45f6-ba22-449294142b56 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:11.252501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:11.253519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:11.254070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:11.254602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:11.262713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:11.264989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:11.278518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:11.279484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:11.281271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:11.281806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:11.281806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:11.281806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:11.281806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:11.282077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:11.282740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:11.284800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 441/901] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:21:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:11.292720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:11.292904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 461/902] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:21:11 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:21:11.304596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:11.415348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc14c60-0994-41bf-a430-a6528ddf7cc2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:11.454805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:11.455428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:11.455707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:11.455981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:11.456787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:11.457095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:11.458609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:11.478073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 442/903] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 27 08:21:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:11.483748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:11.502748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:11.600374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98cc8a-b618-45f4-b045-dba0291d777a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:11.636427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:11.637194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:11.637576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:11.637951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:11.712543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:11.714252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:11.731142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:11.823573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 462/904] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:11.829588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:11.903004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:11.944512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a186-2ef6-4407-b490-02192fdbb433 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:11.981425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:11.982425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:11.982910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:11.983409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:11.993879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:11.995636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:12.011279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:12.018162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:12.019059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:12.019059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:12.019059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:12.019059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:12.019731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 443/905] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:12.023857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:12.132018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586409ca-9ae4-408b-aff8-44005259434b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:12.147074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:12.168358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:12.168984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.169266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.169540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.170268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:12.170677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:12.177718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:12.185175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 463/906] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:12.191160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:12.298183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072626f-20d5-4310-a366-99c9061f4765 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:12.314930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:12.334241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e748a-52ad-4461-a437-c4a59d0462cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:12.334925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e748a-52ad-4461-a437-c4a59d0462cc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.335204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e748a-52ad-4461-a437-c4a59d0462cc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.335476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e748a-52ad-4461-a437-c4a59d0462cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.343435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e748a-52ad-4461-a437-c4a59d0462cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:12.345155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e748a-52ad-4461-a437-c4a59d0462cc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d04e748a-52ad-4461-a437-c4a59d0462cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:12.357274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:12.358406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:12.359626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:12.362258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 444/907] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:12.365150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:12.472178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2aad4f-b6ba-448d-b4d9-702242ceebfc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:12.491902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec79b7e-4bfe-4f97-a76f-5a82b7c2ab4b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.492426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec79b7e-4bfe-4f97-a76f-5a82b7c2ab4b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.492426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec79b7e-4bfe-4f97-a76f-5a82b7c2ab4b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.493773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 445/908] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:12.499511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2175-0aab-44a3-a369-e1805790b888 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.500012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2175-0aab-44a3-a369-e1805790b888 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.500191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2175-0aab-44a3-a369-e1805790b888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.511079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:12.512256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.512686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.513116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.514063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:12.514754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:12.516660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:12.518953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9bde2175-0aab-44a3-a369-e1805790b888 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:12.520253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9bde2175-0aab-44a3-a369-e1805790b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:12.530762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 464/909] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:12.539100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:12.598584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:12.617058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 446/910] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:12.639395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2175-0aab-44a3-a369-e1805790b888 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:12.669308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad0856-1fe5-4034-85ff-b40625b2eff3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:12.710516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d1582-5206-4423-9979-cbff5a69035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:12.711124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d1582-5206-4423-9979-cbff5a69035c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.711410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d1582-5206-4423-9979-cbff5a69035c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.711838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d1582-5206-4423-9979-cbff5a69035c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.712932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d1582-5206-4423-9979-cbff5a69035c None admin] RBAC: Adding query filter params (role.id=7065360498844be996f65d53579e8ab3, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=03d7fd886b4843efa9c022741bb85542) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:12.713253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d1582-5206-4423-9979-cbff5a69035c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:12.714997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d1582-5206-4423-9979-cbff5a69035c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:12.725064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 465/911] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:12 2022] GET /identity/v3/role_assignments?user.id=03d7fd886b4843efa9c022741bb85542&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=7065360498844be996f65d53579e8ab3 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:12.726804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e06e0-e940-4b84-b0a9-4775e3d0962d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.727154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e06e0-e940-4b84-b0a9-4775e3d0962d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.727493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e06e0-e940-4b84-b0a9-4775e3d0962d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.728927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 466/912] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:12.734596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618fed1-5db0-4067-95f0-a625261b7126 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.735202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618fed1-5db0-4067-95f0-a625261b7126 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.735710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618fed1-5db0-4067-95f0-a625261b7126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.755622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6618fed1-5db0-4067-95f0-a625261b7126 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:12.757144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6618fed1-5db0-4067-95f0-a625261b7126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:12.761675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2175-0aab-44a3-a369-e1805790b888 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:12.771032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:12.804221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:12.805206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:12.805690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:12.806194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:12.807129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:12.808864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:12.845399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 467/913] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:12.868575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618fed1-5db0-4067-95f0-a625261b7126 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:12.887268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 447/914] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:21:12 2022] POST /identity/v3/services => generated 241 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:21:12.901687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:12.986345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618fed1-5db0-4067-95f0-a625261b7126 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:13.033568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:13.034298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:13.034651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:13.034925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:13.035564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:13.037201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:13.047555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:13.053098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:13.054406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 468/915] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:13.055206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:13.164743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3012a60-72df-482e-a050-3c00de8a66a8 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:13.200626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:13.201247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:13.201523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:13.202344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:13.203032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:13.203336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:13.204720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:13.210023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 448/916] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:13.215757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:13.312904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:13.325931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5f49f-9e7a-47e3-a6ea-0f336e90ca55 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:13.362344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159ef8d-aec7-48da-8d20-000e813c7dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:13.363235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ef8d-aec7-48da-8d20-000e813c7dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:13.363686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ef8d-aec7-48da-8d20-000e813c7dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:13.364105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ef8d-aec7-48da-8d20-000e813c7dc2 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:13.364927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ef8d-aec7-48da-8d20-000e813c7dc2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:13.365373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ef8d-aec7-48da-8d20-000e813c7dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:13.366896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ef8d-aec7-48da-8d20-000e813c7dc2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:13.372470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 469/917] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 27 08:21:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:13.380382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:13.415580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:13.493553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c2e3-6def-480d-bf6d-0f2d88cf414d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:13.532206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:13.533067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:13.533517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:13.533929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:13.534813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:13.536366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:13.540781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e5149-3d64-4aaa-bec3-3ea182e9bc55 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:13.541269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e5149-3d64-4aaa-bec3-3ea182e9bc55 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:13.541361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e5149-3d64-4aaa-bec3-3ea182e9bc55 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:13.542972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 470/918] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:13.551486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047c31d-ae38-4fc1-81db-8d04e37a13ad None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:13.551486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047c31d-ae38-4fc1-81db-8d04e37a13ad None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:13.551486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9047c31d-ae38-4fc1-81db-8d04e37a13ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:13.568273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9047c31d-ae38-4fc1-81db-8d04e37a13ad None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:13.569479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9047c31d-ae38-4fc1-81db-8d04e37a13ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:13.619501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 449/919] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:13.656098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 471/920] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:13.680638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:13.778752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:13.795111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802d37b-18f7-41b0-921b-fa7932c116be None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:13.830852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:13.831771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:13.832271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:13.832688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:13.833453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:13.835039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:13.845455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:13.846694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 27 08:21:13.847750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:13.850214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 450/921] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 27 08:21:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:13.858520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047c31d-ae38-4fc1-81db-8d04e37a13ad None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:13.940105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:13.961944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9047c31d-ae38-4fc1-81db-8d04e37a13ad None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:13.998124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:13.999821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:14.000065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:14.000336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:14.001017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:14.001317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:14.002732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:14.008321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 472/922] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:14.013993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:14.118064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1ed58-06f5-49f8-b64f-2e1ce038b9d7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:14.154012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fdd66-5d7f-4c37-a454-755a24ccfbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:14.154669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fdd66-5d7f-4c37-a454-755a24ccfbbd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:14.155859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fdd66-5d7f-4c37-a454-755a24ccfbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:14.156133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fdd66-5d7f-4c37-a454-755a24ccfbbd None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:14.156880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fdd66-5d7f-4c37-a454-755a24ccfbbd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:14.157190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fdd66-5d7f-4c37-a454-755a24ccfbbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:14.158597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fdd66-5d7f-4c37-a454-755a24ccfbbd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:14.164356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 451/923] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:14.169970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:14.277981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bf0be-1aee-4368-80bc-5736ffb2b89d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:14.314635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6a39e-8371-48a5-bc42-5290e04c36eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:14.315251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6a39e-8371-48a5-bc42-5290e04c36eb None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:14.315534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6a39e-8371-48a5-bc42-5290e04c36eb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:14.315851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6a39e-8371-48a5-bc42-5290e04c36eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:14.316578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6a39e-8371-48a5-bc42-5290e04c36eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:14.317946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6a39e-8371-48a5-bc42-5290e04c36eb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:14.387907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:14.395604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 473/924] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:14.863568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aa75e-d75c-4b45-938f-058df54bc1fc None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:14.864611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aa75e-d75c-4b45-938f-058df54bc1fc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:14.865109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aa75e-d75c-4b45-938f-058df54bc1fc None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:14.866963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 452/925] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:14.872391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0290-8266-40a9-9eca-917787987501 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:14.873251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0290-8266-40a9-9eca-917787987501 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:14.873772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0290-8266-40a9-9eca-917787987501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:14.893011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-acef0290-8266-40a9-9eca-917787987501 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:14.894652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-acef0290-8266-40a9-9eca-917787987501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:14.972453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 474/926] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:14.994989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aa75e-d75c-4b45-938f-058df54bc1fc None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.012475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:15.072933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4066-ed1e-4253-a187-118199c9ab4a None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.073140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4066-ed1e-4253-a187-118199c9ab4a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.073420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed4066-ed1e-4253-a187-118199c9ab4a None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.075996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 475/927] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:15.080818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37854ae0-9c0a-4fd8-b31b-5ca87c268f5a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.081098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37854ae0-9c0a-4fd8-b31b-5ca87c268f5a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.081372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37854ae0-9c0a-4fd8-b31b-5ca87c268f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.101524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-37854ae0-9c0a-4fd8-b31b-5ca87c268f5a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:15.102928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-37854ae0-9c0a-4fd8-b31b-5ca87c268f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:15.108433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aa75e-d75c-4b45-938f-058df54bc1fc None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.147676 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.148678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.149119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.149566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.162038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:15.167126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:15.168343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:15.169460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:15.172779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 453/928] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 27 08:21:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:15.178980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.208971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 476/929] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:15.231660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37854ae0-9c0a-4fd8-b31b-5ca87c268f5a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.272065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:15.289001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:15.299580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e308c4-bf20-43bb-a1a8-e4246c748828 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.334578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.335194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.335465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.335735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.336507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:15.336810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.337087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37854ae0-9c0a-4fd8-b31b-5ca87c268f5a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.338253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.348311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 454/930] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:15.354901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.382481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.383484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.384024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.384302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.385044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.387061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:15.398612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:15.399991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:15.401845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 477/931] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:15.415540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.475967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d8071-5d56-4919-a122-809b5047f026 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.517298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.518100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.518614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.518990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.522524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:15.534485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.537373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.547138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf80961-6305-4156-b1c9-c0eb9dfe7db4 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:15.553076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:15.554959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:15.557085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:15.557085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:15.557085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:15.557085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:15.560529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 455/932] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:15.565485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.586817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:15.615222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.616687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.617598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.618414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.619326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:15.620080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.622482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.629733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 478/933] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:15.635322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.679522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b3a1a-1c4a-42a3-b18f-799ab0f2c777 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.715683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.716481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.716656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.717065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.718086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:15.718231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.719654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.734188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 456/934] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:15.738095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.771389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5009e-f6f6-4b08-b45d-6ba6faddb6e9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.790227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:15.808723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81c07c-13a6-486c-94ae-3376d1073ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.809952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81c07c-13a6-486c-94ae-3376d1073ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.809952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81c07c-13a6-486c-94ae-3376d1073ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.810218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81c07c-13a6-486c-94ae-3376d1073ca2 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.811613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81c07c-13a6-486c-94ae-3376d1073ca2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:15.811924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81c07c-13a6-486c-94ae-3376d1073ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.814341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81c07c-13a6-486c-94ae-3376d1073ca2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.823296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 479/935] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:15.824683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b0331-9082-415d-a12a-9dd8ceef7567 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.825073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b0331-9082-415d-a12a-9dd8ceef7567 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.825472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b0331-9082-415d-a12a-9dd8ceef7567 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.827744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 480/936] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:15.830679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b0331-9082-415d-a12a-9dd8ceef7567 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:15.852149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe1d2a-e2fc-4232-8fb0-1da0fc303238 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.884228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:15.888124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6218b-982e-49b8-ba1d-717b7e979fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.888940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6218b-982e-49b8-ba1d-717b7e979fdd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.889725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6218b-982e-49b8-ba1d-717b7e979fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.890009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6218b-982e-49b8-ba1d-717b7e979fdd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.897453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6218b-982e-49b8-ba1d-717b7e979fdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.898918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6218b-982e-49b8-ba1d-717b7e979fdd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-97e6218b-982e-49b8-ba1d-717b7e979fdd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:15.911247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:15.912419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:15.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 457/937] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:15.914571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d972c-765d-45a3-a6b2-3c032404e4ec None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.914571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d972c-765d-45a3-a6b2-3c032404e4ec None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.914571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d972c-765d-45a3-a6b2-3c032404e4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.932256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bb8d972c-765d-45a3-a6b2-3c032404e4ec None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:15.933365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bb8d972c-765d-45a3-a6b2-3c032404e4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:15.947073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b0331-9082-415d-a12a-9dd8ceef7567 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:15.984849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:15.985472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:15.985758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:15.986037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:15.986835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] RBAC: Adding query filter params (interface=public, service_id=972ae46d077c456d9ca66a6606e1b094, region_id=RegionOne) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:15.987144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:15.988517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:15.993761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 481/938] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/endpoints?service_id=972ae46d077c456d9ca66a6606e1b094&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:15.994580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.016182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 458/939] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:16.016182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d972c-765d-45a3-a6b2-3c032404e4ec None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.064088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:16.110425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce575a0-7efe-4065-877a-4a448272c37e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.121280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:16.128517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d972c-765d-45a3-a6b2-3c032404e4ec None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.146159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.146800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.147079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.147353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.148146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:16.148453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.149891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.165063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.165685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.165966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.166271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.166931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.172834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 482/940] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 27 08:21:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:16.172976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.173062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.186834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 459/941] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:21:16.186834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.316036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606e52e-ec7d-4c72-8f19-656e97577bc1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.322174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13351fb-1598-42f8-81e1-e580705a1f9b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.323962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:16.361404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.362312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.362594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.362867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.363548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.365019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.367679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING keystone.server.flask.common [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 08:21:16.385218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.385865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.386163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.386490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.398255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.399786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:16.416212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:16.417221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:16.418250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:16.418250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:16.418250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:16.418250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:16.418250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 460/942] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:16.426715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.427640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.sql.core [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 08:21:16.427640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 08:21:16.427640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [parameters: {'id': '77222e15a29d47c8863c5e442c242127', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 08:21:16.427640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80290) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 77222e15a29d47c8863c5e442c242127.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 77222e15a29d47c8863c5e442c242127. Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application conn.query(q) Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 08:21:16.438260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result.read() Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 27 08:21:16.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application next(self.gen) Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application yield resource Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application next(self.gen) Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise self.value Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application session.commit() Jul 27 08:21:16.440155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.session.flush() Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application compat.raise_( Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 08:21:16.441056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 08:21:16.442003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application conn.query(q) Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application result.read() Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 08:21:16.444777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [parameters: {'id': '77222e15a29d47c8863c5e442c242127', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.446887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:16.448827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 77222e15a29d47c8863c5e442c242127. Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 483/943] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Jul 27 08:21:16 2022] POST /identity/v3/projects => generated 262 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:16.451284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.479829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:16.548435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290df5c-87b2-4df3-b105-77168e13fc3c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.564074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4467007-f984-4d68-a6c1-a1a05e943399 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.589485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2cada-c569-4510-bb27-613c1521e257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.590105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2cada-c569-4510-bb27-613c1521e257 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.590412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2cada-c569-4510-bb27-613c1521e257 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.590685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2cada-c569-4510-bb27-613c1521e257 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.591375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2cada-c569-4510-bb27-613c1521e257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:16.591678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2cada-c569-4510-bb27-613c1521e257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.593192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2cada-c569-4510-bb27-613c1521e257 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.605101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 461/944] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:16.606858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcf2d4-83f1-47da-b285-d5e31f4ac579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.607807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf2d4-83f1-47da-b285-d5e31f4ac579 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.608470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf2d4-83f1-47da-b285-d5e31f4ac579 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.608998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcf2d4-83f1-47da-b285-d5e31f4ac579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.611163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2482f3-7094-465b-b694-0807646f4144 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.611468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2482f3-7094-465b-b694-0807646f4144 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.611725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2482f3-7094-465b-b694-0807646f4144 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.616846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 462/945] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:16.616846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2482f3-7094-465b-b694-0807646f4144 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.619573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcf2d4-83f1-47da-b285-d5e31f4ac579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.621239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcf2d4-83f1-47da-b285-d5e31f4ac579 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.631447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-f9fcf2d4-83f1-47da-b285-d5e31f4ac579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:16.633836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:16.634957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:16.636056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 484/946] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:16.640399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb1bb6-35f3-465b-b388-b1df6477d1f8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.640869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb1bb6-35f3-465b-b388-b1df6477d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.641857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb1bb6-35f3-465b-b388-b1df6477d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.665445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d0eb1bb6-35f3-465b-b388-b1df6477d1f8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:16.667550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d0eb1bb6-35f3-465b-b388-b1df6477d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:16.727753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2482f3-7094-465b-b694-0807646f4144 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.743716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 485/947] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:16.751057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb1bb6-35f3-465b-b388-b1df6477d1f8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.767254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:16.770477 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.771131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.771413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.771689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.778753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.780297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:16.791774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:16.792866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:16.793811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 463/948] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:16.794340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.863065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb1bb6-35f3-465b-b388-b1df6477d1f8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.877107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:16.900167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.900837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.901064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.901336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.902063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:16.902411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.903889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.904579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1187-56c0-49e9-a021-f14792fda0f9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:16.925478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 486/949] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:16.926330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:16.952997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:16.953978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:16.954480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:16.955068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:16.955727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:16.957147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:16.969523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:16.971156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:16.972202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:16.974318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 464/950] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:16.978723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:17.031492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:17.061201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfa56c-6db2-433d-ba3e-9144fd321fcf None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:17.104946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:17.105573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:17.105849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:17.106120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:17.106950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:17.107252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:17.108978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:17.111396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d21244-4f29-4245-a64c-473723a64ccf None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:17.122217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 487/951] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:17.128074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:17.152106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:17.153008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:17.153438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:17.153886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:17.154854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] RBAC: Adding query filter params (name=compute) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:17.156036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:17.157561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:17.164250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 465/952] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:17.169981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:17.172827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:17.256242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9994e4-12ad-424a-8830-1e2aad7f0c2d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:17.301300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829f3d8-0538-4495-8db1-aabf720523ca None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:17.301899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f90a4-5222-40e6-ad9a-68bb04c6f53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:17.302669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f90a4-5222-40e6-ad9a-68bb04c6f53e None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:17.303116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f90a4-5222-40e6-ad9a-68bb04c6f53e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:17.303464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f90a4-5222-40e6-ad9a-68bb04c6f53e None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/03d7fd886b4843efa9c022741bb85542/roles/7065360498844be996f65d53579e8ab3` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:17.305882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f90a4-5222-40e6-ad9a-68bb04c6f53e None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=03d7fd886b4843efa9c022741bb85542, role_id=7065360498844be996f65d53579e8ab3)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:17.307540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f90a4-5222-40e6-ad9a-68bb04c6f53e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:17.310263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:17.339621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2916e5-7692-4932-8e7e-17ae2cd2cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:17.340242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2916e5-7692-4932-8e7e-17ae2cd2cc0e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:17.340533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2916e5-7692-4932-8e7e-17ae2cd2cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:17.340787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2916e5-7692-4932-8e7e-17ae2cd2cc0e None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:17.341520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2916e5-7692-4932-8e7e-17ae2cd2cc0e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:17.341824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2916e5-7692-4932-8e7e-17ae2cd2cc0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:17.343982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2916e5-7692-4932-8e7e-17ae2cd2cc0e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:17.349482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 466/953] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:17.396901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 488/954] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:21:17 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/03d7fd886b4843efa9c022741bb85542/roles/7065360498844be996f65d53579e8ab3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:21:18.250917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9eca4-027b-4ec4-a07a-624c69fe93f5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:18.251448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9eca4-027b-4ec4-a07a-624c69fe93f5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:18.251448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9eca4-027b-4ec4-a07a-624c69fe93f5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:18.253072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 467/955] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:18.258476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea932-f469-4987-a194-508791aa75aa None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:18.258893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea932-f469-4987-a194-508791aa75aa None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:18.259042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea932-f469-4987-a194-508791aa75aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:18.277347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-05aea932-f469-4987-a194-508791aa75aa None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:18.278560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-05aea932-f469-4987-a194-508791aa75aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:18.354275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 489/956] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:18.369274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9eca4-027b-4ec4-a07a-624c69fe93f5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:18.436905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:18.474162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9eca4-027b-4ec4-a07a-624c69fe93f5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:18.510714 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:18.512150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:18.512431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:18.512703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:18.513338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:18.515426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:18.524883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:18.525923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:18.526838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 468/957] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:18.532985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea932-f469-4987-a194-508791aa75aa None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:18.640023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea932-f469-4987-a194-508791aa75aa None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:18.667870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:18.679252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:18.679879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:18.680157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:18.684074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:18.684074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:18.684074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:18.684074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:18.688613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 490/958] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:18.694176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:18.799781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1267-1807-4b6a-b0e9-8fdb94871cae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:18.832167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:18.837799 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc253-c840-490d-84a7-15d1a332860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:18.838445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc253-c840-490d-84a7-15d1a332860f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:18.844268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc253-c840-490d-84a7-15d1a332860f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:18.844268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc253-c840-490d-84a7-15d1a332860f None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:18.844268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc253-c840-490d-84a7-15d1a332860f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:18.844268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc253-c840-490d-84a7-15d1a332860f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:18.844268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc253-c840-490d-84a7-15d1a332860f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:18.847542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 469/959] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:18.853344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:18.970969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3cf8e-835f-400d-be17-90b347e467c3 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:19.005903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b817e7-617e-4ef6-af45-d2bbc040c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:19.006559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b817e7-617e-4ef6-af45-d2bbc040c557 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.006968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b817e7-617e-4ef6-af45-d2bbc040c557 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.007238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b817e7-617e-4ef6-af45-d2bbc040c557 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.007967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b817e7-617e-4ef6-af45-d2bbc040c557 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:19.009328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b817e7-617e-4ef6-af45-d2bbc040c557 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:19.152621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 491/960] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:19.172658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226523e2-c2ac-47bb-84a4-3d120390c022 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.173053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226523e2-c2ac-47bb-84a4-3d120390c022 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.173231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226523e2-c2ac-47bb-84a4-3d120390c022 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.175073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 470/961] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:19.180911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a28b0d-2487-4c4d-8bd7-e5d2cdad40a9 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.181228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a28b0d-2487-4c4d-8bd7-e5d2cdad40a9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.181490 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a28b0d-2487-4c4d-8bd7-e5d2cdad40a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.203338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-99a28b0d-2487-4c4d-8bd7-e5d2cdad40a9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:19.204576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-99a28b0d-2487-4c4d-8bd7-e5d2cdad40a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:19.251795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:19.287429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 492/962] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:19.313622 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226523e2-c2ac-47bb-84a4-3d120390c022 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:19.426728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226523e2-c2ac-47bb-84a4-3d120390c022 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:19.462428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:19.463045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.463324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.463595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.464105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21822b2-240d-4754-b953-4f467a9c28f7 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.464281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:19.464746 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21822b2-240d-4754-b953-4f467a9c28f7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.465239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21822b2-240d-4754-b953-4f467a9c28f7 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.466044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:19.472785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING keystone.server.flask.common [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 27 08:21:19.473104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 493/963] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:19.474972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f601431-aedf-4e56-b5ce-3798c5e382db None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.475471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f601431-aedf-4e56-b5ce-3798c5e382db None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.476069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f601431-aedf-4e56-b5ce-3798c5e382db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.496172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4f601431-aedf-4e56-b5ce-3798c5e382db None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:19.497441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4f601431-aedf-4e56-b5ce-3798c5e382db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:19.516315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.sql.core [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 08:21:19.516315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 08:21:19.516315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [parameters: {'user_id': '2598b0d81dd44d549d1174279615fd90', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 08:21:19.516315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80291) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d6c2982a-30bc-4acb-bb8f-971b76baf62b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application conn.query(q) Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result.read() Jul 27 08:21:19.533306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 08:21:19.534481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application next(self.gen) Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application yield resource Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application next(self.gen) Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.session.rollback() Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.force_reraise() Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise self.value Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application session.commit() Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 27 08:21:19.540634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._prepare_impl() Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.session.flush() Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._flush(objects) Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application compat.raise_( Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application flush_context.execute() Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rec.execute(self) Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 27 08:21:19.541589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 27 08:21:19.542546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result = self._query(query) Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application conn.query(q) Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result.read() Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 27 08:21:19.543415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [parameters: {'user_id': '2598b0d81dd44d549d1174279615fd90', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:19.551212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 27 08:21:19.553724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 471/964] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 27 08:21:19 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f2935-f180-4ff2-bfea-bfa73455b39e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f2935-f180-4ff2-bfea-bfa73455b39e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.556942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f2935-f180-4ff2-bfea-bfa73455b39e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.558374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 472/965] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:19.558374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f2935-f180-4ff2-bfea-bfa73455b39e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:19.585046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 494/966] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:19.594307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec68161-84fe-4ce4-9cdd-fe517a0daf76 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.594307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec68161-84fe-4ce4-9cdd-fe517a0daf76 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.594307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec68161-84fe-4ce4-9cdd-fe517a0daf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.605567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5ec68161-84fe-4ce4-9cdd-fe517a0daf76 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:19.607227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5ec68161-84fe-4ce4-9cdd-fe517a0daf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:19.619742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:19.669923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f2935-f180-4ff2-bfea-bfa73455b39e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:19.683322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 495/967] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:19.684075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec68161-84fe-4ce4-9cdd-fe517a0daf76 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:19.719105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:19.720779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:19.721420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.721696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.721969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.730017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:19.735049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:19.744029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:19.744990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 473/968] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:21:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:19.751008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:19.805574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec68161-84fe-4ce4-9cdd-fe517a0daf76 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:19.841362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:19.842061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.842378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.842808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.843586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:19.845256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:19.860243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d03d45-26ff-4f4b-b82b-bd9d0c1dccff None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:19.876776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:19.894999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:19.895674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:19.895954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:19.896227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:19.907898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:19.912754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:19.913665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:19.914605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:19.918757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 474/969] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 27 08:21:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:19.919482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:19.948571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 496/970] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:21:19 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:19.949625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:20.033666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24be811-31f9-412b-a6b6-ece8ca8868d7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:20.060118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:20.078823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bed95-04fa-4f48-9b28-dfa2e49e8eb5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:20.079183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:20.079721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:20.080002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:20.080270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:20.081032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:20.081329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:20.082821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:20.120902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:20.121698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 475/971] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 27 08:21:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:20.121833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:20.121833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:20.122101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:20.122864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:20.123259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:20.125127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:20.135574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 497/972] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 27 08:21:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:20.142781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:20.185113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:20.266599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b1477-1c9a-42f8-a7c9-e8d82616458f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:20.307055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:20.307709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:20.307989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:20.308264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:20.319576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:20.321038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:20.336804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:20.337769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:20.339276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:20.339276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:20.339276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:20.339276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:20.339276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 476/973] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:20.344471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:20.453239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab219b-e47a-486d-b1f2-4bf485198e52 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:20.486775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:20.487390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:20.487667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:20.487940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:20.488630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:20.488933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:20.490327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:20.503581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 498/974] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:20.509213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:20.599756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:20.617639 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e387bc-0f57-485b-aa31-f1f5416bd2ba None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:20.653749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:20.655052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:20.655680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:20.656201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:20.664052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:20.665744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:20.677837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:20.679063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 27 08:21:20.680228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:20.687523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 477/975] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:21:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:20.688652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:20.700176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:20.791984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51117c2f-26b0-47b7-ac0f-7cef4eb5692b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:20.826090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:20.827400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:20.827772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:20.828172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:20.829062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:20.829468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:20.831020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:20.853548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 499/976] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 27 08:21:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:20.854985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:20.956316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256be45-da6b-47d9-b4a7-9acf7928995e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:20.989831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163a5dc-576d-4dbf-b27f-ff00afcc1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:20.990629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163a5dc-576d-4dbf-b27f-ff00afcc1473 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:20.991009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163a5dc-576d-4dbf-b27f-ff00afcc1473 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:20.991384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163a5dc-576d-4dbf-b27f-ff00afcc1473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.003934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.061157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163a5dc-576d-4dbf-b27f-ff00afcc1473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.062792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163a5dc-576d-4dbf-b27f-ff00afcc1473 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.166276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 478/977] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:20 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 27 08:21:21.168131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.239456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.250493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2646d2-af15-4918-a0b8-cd0c8563d072 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.250772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2646d2-af15-4918-a0b8-cd0c8563d072 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.251042 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2646d2-af15-4918-a0b8-cd0c8563d072 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.252531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 479/978] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:21.257114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f9afc-c8ee-47c1-8eea-14b939ce618a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.257395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f9afc-c8ee-47c1-8eea-14b939ce618a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.257668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f9afc-c8ee-47c1-8eea-14b939ce618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.275834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901569-c545-4003-9e30-8009ee7132ac None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.276032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-541f9afc-c8ee-47c1-8eea-14b939ce618a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:21.277363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-541f9afc-c8ee-47c1-8eea-14b939ce618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:21.312875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:21.313511 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.313795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.314070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.315070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.325683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.327891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:21.343158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:21.344164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:21.345814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:21.345814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:21.345814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:21.345814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:21.345814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 500/979] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:21.350061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.354952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 480/980] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:21.372466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f9afc-c8ee-47c1-8eea-14b939ce618a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.459287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35bb8-8e00-41a0-b18a-400b4f3b0041 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.477758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f9afc-c8ee-47c1-8eea-14b939ce618a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.480314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.498190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:21.498805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.505214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.505214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.505214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:21.505214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.505214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.513060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:21.514234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.514528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.514803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.515473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.516847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.518109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 501/981] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:21.523682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:21.526001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:21.527011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:21.527873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 481/982] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:21.533646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.578977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.640313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fe2ba-1e05-4d3d-a609-06713be1ad08 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.649821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e00e99-6cd5-4340-a1af-0731ff500cf5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.677865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:21.678528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.678841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.679071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.687205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.688653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.689046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:21.689668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.689952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.690293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.690979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:21.691299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.692641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.698173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 482/983] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:21.703755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:21.705526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:21.706640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:21.707596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 502/984] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:21.713154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.756128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.811569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.815026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa9a5a-885d-4187-8f26-ffa27b84e6de None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.819315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ecf06-3f85-420a-9aa4-752dbe2f1d7c None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.851915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:21.852553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.852827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.853095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.853760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:21.854060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.855467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.859396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:21.861089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:21.861089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:21.861089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:21.864111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:21.864111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:21.864111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:21.866314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 483/985] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:21.870078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.882930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 503/986] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:21.888301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:21.974977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653eeb1-9449-4964-98a9-77d90c4b16d9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:21.989547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:21.993469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f91ab9-d229-482d-91f2-c64d59a5a6b5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:22.010343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.010958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.011230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.011700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.012250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] RBAC: Adding query filter params (interface=admin, service_id=972ae46d077c456d9ca66a6606e1b094, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:22.012558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.014600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:22.025152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 484/987] 10.222.0.40 () {56 vars in 1249 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/endpoints?service_id=972ae46d077c456d9ca66a6606e1b094&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:22.026184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:22.034085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b02ce-e414-44e9-802a-71e7234f5291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.034747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b02ce-e414-44e9-802a-71e7234f5291 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.035028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b02ce-e414-44e9-802a-71e7234f5291 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.035297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b02ce-e414-44e9-802a-71e7234f5291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.036086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b02ce-e414-44e9-802a-71e7234f5291 None admin] RBAC: Adding query filter params (role.id=7065360498844be996f65d53579e8ab3, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=03d7fd886b4843efa9c022741bb85542) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:22.036394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b02ce-e414-44e9-802a-71e7234f5291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.037946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05b02ce-e414-44e9-802a-71e7234f5291 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:22.054200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 504/988] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:21 2022] GET /identity/v3/role_assignments?user.id=03d7fd886b4843efa9c022741bb85542&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=7065360498844be996f65d53579e8ab3 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:22.057749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:22.066561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981ae59-ecf0-45ba-95f1-a9a3a22aa75e None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.072586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981ae59-ecf0-45ba-95f1-a9a3a22aa75e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.072586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981ae59-ecf0-45ba-95f1-a9a3a22aa75e None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.072586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 505/989] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:22.074296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4e488-a0f2-40b5-bb66-42a26771b925 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.074590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4e488-a0f2-40b5-bb66-42a26771b925 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.074870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4e488-a0f2-40b5-bb66-42a26771b925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.095576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d0f4e488-a0f2-40b5-bb66-42a26771b925 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:22.096818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d0f4e488-a0f2-40b5-bb66-42a26771b925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:22.147991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881237d4-b326-41e1-a9cf-44f997d272ec None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:22.177697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 506/990] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:22.186084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.186737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.187013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.187284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.187943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.189340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:22.202407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 485/991] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:21:22 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:21:22.202614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4e488-a0f2-40b5-bb66-42a26771b925 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:22.307109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:22.316889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4e488-a0f2-40b5-bb66-42a26771b925 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:22.354916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.355549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.355826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.356538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.356721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.358472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:22.367815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:22.368814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:22.369636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 507/992] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:22.375535 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:22.471702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:22.480600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0e559-d368-4301-99e8-1c99b4082ae0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:22.515178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.515797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.516078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.516348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.517028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:22.517329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.518724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:22.523642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 486/993] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:22.530537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:22.641544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16470e-8a58-486e-a9f4-266dbc79973a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:22.675769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.676390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.676670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.676944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.677623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] RBAC: Adding query filter params (type=compute) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:22.677935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.679338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:22.689754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 508/994] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:22.691278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:22.793824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa191766-9be3-45b1-957f-58a581461203 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:22.828736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c8c4b-d576-4fc6-970f-707e595c309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.829400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c8c4b-d576-4fc6-970f-707e595c309d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.829626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c8c4b-d576-4fc6-970f-707e595c309d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.829945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c8c4b-d576-4fc6-970f-707e595c309d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.830666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c8c4b-d576-4fc6-970f-707e595c309d None admin] RBAC: Adding query filter params (interface=public, service_id=7e1c56fa3c3c49d686569ef3ff06a359, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:22.830975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c8c4b-d576-4fc6-970f-707e595c309d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.832329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c8c4b-d576-4fc6-970f-707e595c309d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:22.841998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 487/995] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:21:22 2022] GET /identity/v3/endpoints?service_id=7e1c56fa3c3c49d686569ef3ff06a359&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:22.843306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:22.868927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:22.950537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b50513-03be-49b1-970f-534ebaa7beae None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:22.984488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ee470-bff8-48a1-a2ca-34fd7a75dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:22.985104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ee470-bff8-48a1-a2ca-34fd7a75dbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:22.985384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ee470-bff8-48a1-a2ca-34fd7a75dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:22.985712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ee470-bff8-48a1-a2ca-34fd7a75dbb2 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:22.986395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ee470-bff8-48a1-a2ca-34fd7a75dbb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:22.987764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ee470-bff8-48a1-a2ca-34fd7a75dbb2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:23.002172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 509/996] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:21:22 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:21:24.123422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719b35-8db3-4e5f-9802-54e16450d912 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.123946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719b35-8db3-4e5f-9802-54e16450d912 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.123946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719b35-8db3-4e5f-9802-54e16450d912 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.125457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 488/997] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:24.130390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec2602-0d44-430a-be15-2d58a6902c96 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.130790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec2602-0d44-430a-be15-2d58a6902c96 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.130974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec2602-0d44-430a-be15-2d58a6902c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.148876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-02ec2602-0d44-430a-be15-2d58a6902c96 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:24.150056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-02ec2602-0d44-430a-be15-2d58a6902c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:24.210963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5623-24b5-4d7d-bda1-3c1653538e3e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.211236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5623-24b5-4d7d-bda1-3c1653538e3e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.211516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe5623-24b5-4d7d-bda1-3c1653538e3e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.213002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 489/998] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:24.218213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c022c-a30d-413d-8425-56129dc6cdc2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.218498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c022c-a30d-413d-8425-56129dc6cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.218767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c022c-a30d-413d-8425-56129dc6cdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.221267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 510/999] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:24.238631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6a6c022c-a30d-413d-8425-56129dc6cdc2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:24.239809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6a6c022c-a30d-413d-8425-56129dc6cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:24.243354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec2602-0d44-430a-be15-2d58a6902c96 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:24.254574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:24.259145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:24.309977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 490/1000] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:24.332265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c022c-a30d-413d-8425-56129dc6cdc2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:24.343975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec2602-0d44-430a-be15-2d58a6902c96 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:24.379596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:24.380155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.380433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.380704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] PATH_INFO: `/v3/services/image` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.381324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:24.382726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:24.392191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:24.393165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:24.394556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 511/1001] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 27 08:21:24 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:24.399174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:24.434889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c022c-a30d-413d-8425-56129dc6cdc2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:24.470067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:24.470776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.471055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.471326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.471941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:24.473291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:24.482238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:24.483518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 27 08:21:24.485038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:24.485790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:24.488426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 491/1002] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 27 08:21:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:24.489280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:24.506098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:24.508676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfa156-5e3d-4b56-9376-9c9958429749 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:24.546900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:24.547522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.547798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.548069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.551524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] RBAC: Adding query filter params (name=image) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:24.551524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:24.551524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:24.556103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 512/1003] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:24.561461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:24.605919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba579609-b2c6-4e77-8ce2-f2d0db6160c7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:24.648528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:24.649063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.649339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.649607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.650715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:24.651153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:24.652785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:24.659823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 492/1004] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:24.662638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2fc69-0efe-41b9-b7c2-4ae9e4538a90 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:24.665644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:24.700160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:24.700788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.701629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.701899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.702768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] RBAC: Adding query filter params (type=image) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:24.703856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:24.705229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:24.722175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 513/1005] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:24.751695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:24.777097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191f04d-9a1b-422e-9c89-aeb7b120ecb8 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:24.814235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0efd5-af36-411d-8503-5b83bdb08039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:24.815000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0efd5-af36-411d-8503-5b83bdb08039 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.815286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0efd5-af36-411d-8503-5b83bdb08039 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.815560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0efd5-af36-411d-8503-5b83bdb08039 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.816251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0efd5-af36-411d-8503-5b83bdb08039 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:24.816558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0efd5-af36-411d-8503-5b83bdb08039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:24.817939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0efd5-af36-411d-8503-5b83bdb08039 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:24.828295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 493/1006] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 27 08:21:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:24.830196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:24.839925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:24.941110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67148e73-db60-43fd-9ab6-64e923604d97 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:24.974695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:24.975314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:24.975594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:24.975867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:24.976586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:24.977959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:25.017333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221872a9-9e5d-46ac-82f7-d6b9d92dff4a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:25.017693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221872a9-9e5d-46ac-82f7-d6b9d92dff4a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:25.018086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221872a9-9e5d-46ac-82f7-d6b9d92dff4a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:25.019341 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 494/1007] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:25.024362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bdbc2-45aa-4d27-9310-ce25a81ed1f3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:25.028440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bdbc2-45aa-4d27-9310-ce25a81ed1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:25.028440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bdbc2-45aa-4d27-9310-ce25a81ed1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:25.042748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-258bdbc2-45aa-4d27-9310-ce25a81ed1f3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:25.043936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-258bdbc2-45aa-4d27-9310-ce25a81ed1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:25.066170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 514/1008] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:21:25.134530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 495/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:25.153678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:25.228743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:25.266926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e617dc2-b49b-4c3a-af44-1874c91d656d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:25.301376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:25.302022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:25.302332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:25.302604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:25.303225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:25.304648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:25.313751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:25.314759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:25.315591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 515/1010] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:25.320763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bdbc2-45aa-4d27-9310-ce25a81ed1f3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:25.389379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:25.422812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bdbc2-45aa-4d27-9310-ce25a81ed1f3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:25.458820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:25.459441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:25.459722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:25.459994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:25.460664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] RBAC: Adding query filter params (name=compute) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:25.460968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:25.462343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:25.467333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 496/1011] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:25.473075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:25.582261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf1e8b-55a5-4ee6-9e78-d7a64756d29a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:25.611198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:25.617315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67fb4c-6b2e-4fb3-bb28-3e2172dd4af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:25.617923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67fb4c-6b2e-4fb3-bb28-3e2172dd4af5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:25.618228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67fb4c-6b2e-4fb3-bb28-3e2172dd4af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:25.618503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67fb4c-6b2e-4fb3-bb28-3e2172dd4af5 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:25.619243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67fb4c-6b2e-4fb3-bb28-3e2172dd4af5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:25.619542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67fb4c-6b2e-4fb3-bb28-3e2172dd4af5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:25.620909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67fb4c-6b2e-4fb3-bb28-3e2172dd4af5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:25.630808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 516/1012] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 27 08:21:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:25.631839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:25.734525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9187ad1-ef94-4e32-bcbe-9f2a9c2c80ae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:25.764219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:25.769285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331b9ff-4ed6-4c8b-a1ec-fbca3f8163aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:25.769908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b9ff-4ed6-4c8b-a1ec-fbca3f8163aa None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:25.770210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b9ff-4ed6-4c8b-a1ec-fbca3f8163aa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:25.770489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b9ff-4ed6-4c8b-a1ec-fbca3f8163aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:25.771211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331b9ff-4ed6-4c8b-a1ec-fbca3f8163aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:25.772586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331b9ff-4ed6-4c8b-a1ec-fbca3f8163aa None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:25.874173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 497/1013] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:25 2022] POST /identity/v3/endpoints => generated 334 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:21:26.773588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c629ee-6535-4f2a-a68b-1b31508870a0 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:26.774097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c629ee-6535-4f2a-a68b-1b31508870a0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:26.774199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c629ee-6535-4f2a-a68b-1b31508870a0 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:26.775647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 517/1014] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:26.780591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c958b-45f1-456b-9194-addbc7882d81 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:26.780977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c958b-45f1-456b-9194-addbc7882d81 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:26.781139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c958b-45f1-456b-9194-addbc7882d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:26.800751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-319c958b-45f1-456b-9194-addbc7882d81 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:26.801953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-319c958b-45f1-456b-9194-addbc7882d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:26.874787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 498/1015] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:26 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:26.896210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c629ee-6535-4f2a-a68b-1b31508870a0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:26.998667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c629ee-6535-4f2a-a68b-1b31508870a0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:27.035690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:27.036305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:27.036586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:27.036856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:27.037555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:27.038949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:27.108071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67c46c-b468-4b4a-b2af-96ad22b3fec3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:27.108356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67c46c-b468-4b4a-b2af-96ad22b3fec3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:27.108631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67c46c-b468-4b4a-b2af-96ad22b3fec3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:27.110052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 499/1016] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:27.114888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2ca58-e83d-403d-a656-64fe51558eca None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:27.115172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2ca58-e83d-403d-a656-64fe51558eca None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:27.115443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2ca58-e83d-403d-a656-64fe51558eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:27.130347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 518/1017] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:21:26 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:27.135034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eec2ca58-e83d-403d-a656-64fe51558eca None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:27.136262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eec2ca58-e83d-403d-a656-64fe51558eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:27.219072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 500/1018] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:27.242558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:27.304925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:27.351409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404fb27-5173-40f6-97e7-8a9307d90a0d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:27.386118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c13fd9-500f-46d2-9131-1c162f5cee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:27.386709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13fd9-500f-46d2-9131-1c162f5cee8e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:27.386985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13fd9-500f-46d2-9131-1c162f5cee8e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:27.387249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13fd9-500f-46d2-9131-1c162f5cee8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:27.388937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13fd9-500f-46d2-9131-1c162f5cee8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:27.390406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13fd9-500f-46d2-9131-1c162f5cee8e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:27.392931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 519/1019] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:27.398557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2ca58-e83d-403d-a656-64fe51558eca None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:27.442610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:27.500789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2ca58-e83d-403d-a656-64fe51558eca None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:27.535597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51600123-a0ea-4452-be94-dc5a2a9d5366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:27.536211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51600123-a0ea-4452-be94-dc5a2a9d5366 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:27.536487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51600123-a0ea-4452-be94-dc5a2a9d5366 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:27.536781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51600123-a0ea-4452-be94-dc5a2a9d5366 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:27.537465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51600123-a0ea-4452-be94-dc5a2a9d5366 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:27.538922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51600123-a0ea-4452-be94-dc5a2a9d5366 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:27.619166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 501/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:21:27 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:27.971269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac349e-0d39-4e0c-8f85-b8c2756d7cb7 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:27.971839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac349e-0d39-4e0c-8f85-b8c2756d7cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:27.971839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac349e-0d39-4e0c-8f85-b8c2756d7cb7 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:27.973318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 520/1021] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:27.978171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba531-87b2-4f8a-8e9b-ee337d5dde88 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:27.978550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba531-87b2-4f8a-8e9b-ee337d5dde88 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:27.978730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba531-87b2-4f8a-8e9b-ee337d5dde88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:27.996490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-00dba531-87b2-4f8a-8e9b-ee337d5dde88 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:27.998208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-00dba531-87b2-4f8a-8e9b-ee337d5dde88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:28.065105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 502/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:28.087540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac349e-0d39-4e0c-8f85-b8c2756d7cb7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:28.194721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac349e-0d39-4e0c-8f85-b8c2756d7cb7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:28.217412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:28.231659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:28.231659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:28.231659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:28.231659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:28.242168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:28.247101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:28.248888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:28.249886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 521/1023] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 27 08:21:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:28.254994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba531-87b2-4f8a-8e9b-ee337d5dde88 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:28.281421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:28.356889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba531-87b2-4f8a-8e9b-ee337d5dde88 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:28.391765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:28.392376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:28.392652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:28.392920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:28.393619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:28.393919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:28.395306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:28.403521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 503/1024] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 27 08:21:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:28.409346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:28.507857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426f641-6989-4cbc-9476-676f8c0cc5ae None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:28.541425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:28.542052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:28.542355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:28.542625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:28.556842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:28.558307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:28.575664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:28.576587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:28.577472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:28.577472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:28.577472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:28.577472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:28.579090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 522/1025] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:28.585182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:28.673190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:28.688514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a218582-83ca-4c6a-ba99-336d5930425e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:28.723128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:28.723748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:28.724028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:28.724356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:28.725052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:28.725357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:28.726758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:28.740377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 504/1026] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:28.745872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:28.782492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:28.851318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9c894-ddee-4f12-a841-7deda7f46660 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:28.886716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:28.887356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:28.887634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:28.887929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:28.895947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:28.897382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:28.908859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:28.909777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:21:28.910740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:28.914954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 523/1027] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:21:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:28.916326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.017605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02dd39-8cba-4671-90b6-b4f89e7f00e5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.052212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa8255-41e0-4496-b0db-de903054c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.052754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8255-41e0-4496-b0db-de903054c700 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.053027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8255-41e0-4496-b0db-de903054c700 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.053297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8255-41e0-4496-b0db-de903054c700 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.054079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa8255-41e0-4496-b0db-de903054c700 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:29.054506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa8255-41e0-4496-b0db-de903054c700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.055812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa8255-41e0-4496-b0db-de903054c700 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.078001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 505/1028] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:21:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:29.079748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.173554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13aeb5-81c4-4869-b86b-75fe8fb5e096 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.173838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13aeb5-81c4-4869-b86b-75fe8fb5e096 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.174108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13aeb5-81c4-4869-b86b-75fe8fb5e096 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.175693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 506/1029] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:29.180475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c52d-9147-4647-90d4-63939f909c43 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.180759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c52d-9147-4647-90d4-63939f909c43 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.181032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c52d-9147-4647-90d4-63939f909c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.181479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.187564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.188444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec67c5d-b01f-4644-ac39-387a7dd8d315 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.199187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0c13c52d-9147-4647-90d4-63939f909c43 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:29.200381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0c13c52d-9147-4647-90d4-63939f909c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:29.227304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.227926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.228201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.228484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.282523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 507/1030] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:29.299246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.300694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.305198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13c52d-9147-4647-90d4-63939f909c43 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.401600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.411857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 524/1031] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:29.413591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13c52d-9147-4647-90d4-63939f909c43 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.417542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.447948 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.453616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.454371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.454650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.455707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] PATH_INFO: `/v3/services/image` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.455707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.456991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:29.466507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:29.467625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:29.468467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 508/1032] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:29.474119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.524901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fec63e-83e0-43da-9ee6-dfa60851b952 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.564815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.565379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.565656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.565931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.576865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.578330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.589712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9e743-a65e-403a-afaa-36b4b0e6be9c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:29.593723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:29.595074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:29.596172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:29.596172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:29.596172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:29.596172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:29.599721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 525/1033] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:29.600531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.632930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.633927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.634390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.634668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.635345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] RBAC: Adding query filter params (name=image) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:29.635651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.637116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.647503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 509/1034] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:29.648721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.654846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.711947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.714010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fe7a1-b641-4eae-b7eb-a48cd447b988 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.748534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbf66a-1980-4250-a7cb-96c8ec41869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.749145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbf66a-1980-4250-a7cb-96c8ec41869a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.749425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbf66a-1980-4250-a7cb-96c8ec41869a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.749696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbf66a-1980-4250-a7cb-96c8ec41869a None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.750416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbf66a-1980-4250-a7cb-96c8ec41869a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:29.750718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbf66a-1980-4250-a7cb-96c8ec41869a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.752083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbf66a-1980-4250-a7cb-96c8ec41869a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.753681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91a124-7046-4e1d-9b22-8a918b330a9c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.764197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 526/1035] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:29.765385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305248e-4ff8-4921-b1b9-e43b724aacba None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.765721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305248e-4ff8-4921-b1b9-e43b724aacba None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.766000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305248e-4ff8-4921-b1b9-e43b724aacba None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.767425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 527/1036] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:29.769799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305248e-4ff8-4921-b1b9-e43b724aacba None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.789121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2424c6-dba8-40af-ab9d-76a4526fe332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.789759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2424c6-dba8-40af-ab9d-76a4526fe332 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.790033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2424c6-dba8-40af-ab9d-76a4526fe332 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.790332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2424c6-dba8-40af-ab9d-76a4526fe332 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.791003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2424c6-dba8-40af-ab9d-76a4526fe332 None admin] RBAC: Adding query filter params (type=image) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:29.791303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2424c6-dba8-40af-ab9d-76a4526fe332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.792647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2424c6-dba8-40af-ab9d-76a4526fe332 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.797350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 510/1037] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:29.798514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714809c-31d0-4bad-bbe9-156bacba9768 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.798793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714809c-31d0-4bad-bbe9-156bacba9768 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.799063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714809c-31d0-4bad-bbe9-156bacba9768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.816615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9714809c-31d0-4bad-bbe9-156bacba9768 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:29.817721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9714809c-31d0-4bad-bbe9-156bacba9768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:29.875600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305248e-4ff8-4921-b1b9-e43b724aacba None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:29.882640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.883044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 511/1038] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:29.883981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714809c-31d0-4bad-bbe9-156bacba9768 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.919622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:29.920273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:29.920552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:29.920825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:29.927844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:29.929280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:29.938919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:29.940896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:29.941799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:29.942751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 528/1039] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:29.943233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:29.988389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714809c-31d0-4bad-bbe9-156bacba9768 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:30.024144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:30.024777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:30.025022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:30.025721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:30.026520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] RBAC: Adding query filter params (interface=public, service_id=13da8026362b465c96a465f98492795c, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:30.026888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:30.028242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:30.033509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 512/1040] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/endpoints?service_id=13da8026362b465c96a465f98492795c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:30.034316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:30.040977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db75610-97bc-4808-b0a4-b86df9da32f1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:30.080144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:30.080787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:30.081050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:30.081396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:30.082559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:30.083995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:30.085581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 529/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:30.086371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:30.120768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:30.136781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857971-a0de-4fc2-aabe-be753349a552 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:30.154491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:30.170974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:30.171596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:30.171878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:30.172153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:30.172901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:30.173208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:30.174707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:30.185861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ea7b-6c41-4768-971e-c131996f99db None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:30.191562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 513/1042] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:30.191562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:30.220586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:30.221203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:30.221485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:30.221764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:30.222442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:30.223812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:30.230229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 530/1043] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:21:30 2022] GET /identity/v3/services => generated 1957 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:21:30.231004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:30.302065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c21b-5b77-4542-9795-472d1d07547b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:30.339518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b02b4a-3ab7-43af-ad45-fe371dbda50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:30.340069 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b02b4a-3ab7-43af-ad45-fe371dbda50c None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:30.340354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b02b4a-3ab7-43af-ad45-fe371dbda50c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:30.340626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b02b4a-3ab7-43af-ad45-fe371dbda50c None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:30.341388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b02b4a-3ab7-43af-ad45-fe371dbda50c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:30.343400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b02b4a-3ab7-43af-ad45-fe371dbda50c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:30.361139 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:30.365389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d173b1-a07b-4b74-8dae-34d8df4e002c None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:30.400582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3a7b5-fb52-4712-81eb-740715cfd5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:30.401281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3a7b5-fb52-4712-81eb-740715cfd5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:30.401562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3a7b5-fb52-4712-81eb-740715cfd5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:30.401837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3a7b5-fb52-4712-81eb-740715cfd5b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:30.402687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3a7b5-fb52-4712-81eb-740715cfd5b5 None admin] RBAC: Adding query filter params (role.id=7065360498844be996f65d53579e8ab3, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e4e461b0b6142b38fd7de56395733b2) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:30.403003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3a7b5-fb52-4712-81eb-740715cfd5b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:30.404993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc3a7b5-fb52-4712-81eb-740715cfd5b5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:30.410534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 531/1044] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:30 2022] GET /identity/v3/role_assignments?user.id=4e4e461b0b6142b38fd7de56395733b2&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=7065360498844be996f65d53579e8ab3 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:30.426273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 514/1045] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:21:30 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:32.262331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fed38-71a4-45a7-b00f-dbd3b09df2f7 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.262854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fed38-71a4-45a7-b00f-dbd3b09df2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.262854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fed38-71a4-45a7-b00f-dbd3b09df2f7 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.264978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 532/1046] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:32.269979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43181bc-e331-42c5-a655-8a6c467c5e17 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.270395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43181bc-e331-42c5-a655-8a6c467c5e17 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.270573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43181bc-e331-42c5-a655-8a6c467c5e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.288856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f43181bc-e331-42c5-a655-8a6c467c5e17 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:32.290034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f43181bc-e331-42c5-a655-8a6c467c5e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:32.318259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:32.362221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 515/1047] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:32.383126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fed38-71a4-45a7-b00f-dbd3b09df2f7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:32.425918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57d4e-10ee-45eb-88e6-6d5909ca98b1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.426266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57d4e-10ee-45eb-88e6-6d5909ca98b1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.426514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b57d4e-10ee-45eb-88e6-6d5909ca98b1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.428019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 516/1048] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:32.432608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6cb5-3ab8-4336-bde7-703f10d57394 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.432895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6cb5-3ab8-4336-bde7-703f10d57394 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.433170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6cb5-3ab8-4336-bde7-703f10d57394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.453281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b58d6cb5-3ab8-4336-bde7-703f10d57394 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:32.454498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b58d6cb5-3ab8-4336-bde7-703f10d57394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:32.491840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fed38-71a4-45a7-b00f-dbd3b09df2f7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:32.524570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 517/1049] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:32.527525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0627c-5e1e-4d40-aeb8-74498b770637 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.527816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0627c-5e1e-4d40-aeb8-74498b770637 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.528093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0627c-5e1e-4d40-aeb8-74498b770637 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.529641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:32.529885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 518/1050] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:32.533241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.533241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.533241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] PATH_INFO: `/v3/services/image` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.533241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:32.533241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:32.535218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d12c6-62d3-44fc-bccf-9c4f8c29caef None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.535502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d12c6-62d3-44fc-bccf-9c4f8c29caef None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.535777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d12c6-62d3-44fc-bccf-9c4f8c29caef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:32.546401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:32.548042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 27 08:21:32.549122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:32.552355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 533/1051] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 27 08:21:32 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:32.553720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:32.556529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e22d12c6-62d3-44fc-bccf-9c4f8c29caef None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:32.557721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e22d12c6-62d3-44fc-bccf-9c4f8c29caef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:32.584799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:32.626134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 519/1052] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:32.626995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d12c6-62d3-44fc-bccf-9c4f8c29caef None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:32.655416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:32.658729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f71c1-35f1-4e12-acdc-5889672e7102 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:32.692512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:32.693135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.693415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.693687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.703573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:32.708853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:32.709968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 534/1053] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 27 08:21:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:32.711059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:32.732787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d12c6-62d3-44fc-bccf-9c4f8c29caef None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:32.765997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:32.766638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.766913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.767182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.767850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:32.768150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:32.769496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:32.774173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 520/1054] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:32.775003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:32.799158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:32.813517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a64881-8915-40a8-b035-749f9a0ce8e7 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:32.848818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:32.849446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.849758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.850030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.859787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:32.860810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:32.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:32.873210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 535/1055] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:32.874054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:32.874858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39cf79-972e-4408-a4b0-bc52f5992ab6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:32.907340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:32.907959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:32.908253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:32.908505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:32.909203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:32.909499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:32.910888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:32.923286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 521/1056] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 27 08:21:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:32.923286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:32.967095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc41422-9941-4c5b-ab2f-73ee286e7491 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.000244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.000935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.001214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.001486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.002178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:33.002480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.003837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.008788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 536/1057] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:33.009371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.021809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.022293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b75167-774f-4be9-8e6e-97eb95c10793 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.056201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.056740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.057039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.057360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.058076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:33.058412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.060337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.069106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 522/1058] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:21:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:33.070197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.086004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.109943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb722-8281-4e8d-81ab-509ff97e35d6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.144853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.145436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.145723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.145995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.158378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.159998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.173171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67598cb-5a69-463c-9f4f-05d6dd9035b4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:33.173994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:33.175040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:33.175991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:33.175991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:33.175991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:33.175991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:33.175991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 537/1059] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:33.176269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.212503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.213111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.213410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.213709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.214439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.215824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.240920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.278525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ed3e9-aca8-4bfd-a792-3e4719663ca8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.296283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 523/1060] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:21:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:33.297212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.322989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.322989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.322989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.322989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] PATH_INFO: `/v3/users/f8c655bdad6a4a70a131eb7884b223ee` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.338742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c655bdad6a4a70a131eb7884b223ee)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.340448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.347735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 538/1061] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/users/f8c655bdad6a4a70a131eb7884b223ee => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:33.349023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.396424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.434775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd1134-e574-4c58-beda-838a9e523c0d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.464448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e4a0d-17eb-47da-a352-8d98d2aa9e58 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.477221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.478025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.478418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.478698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.479779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:33.480239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.481608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.482322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.495564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 524/1062] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:33.502469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.504344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e413711-e058-4068-b886-85e85323a654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.505004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e413711-e058-4068-b886-85e85323a654 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.505245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e413711-e058-4068-b886-85e85323a654 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.505516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e413711-e058-4068-b886-85e85323a654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.583666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e413711-e058-4068-b886-85e85323a654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.585783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e413711-e058-4068-b886-85e85323a654 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.609619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ece9bf-4468-4a16-9f05-6d7da7bbfe65 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.641311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.649642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.650544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.651045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.651318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.659452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.660889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:33.673376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:33.674601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:21:33.675860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:33.679763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 525/1063] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:33.680594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.697477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 539/1064] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:33.703440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e413711-e058-4068-b886-85e85323a654 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.720905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.788383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a428a1-d8aa-4071-a9b9-c33d06f0ea59 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.815033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e413711-e058-4068-b886-85e85323a654 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.823406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.824076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.824361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.824641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.826068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:33.826475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.827912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.845314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 526/1065] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:33.850823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.857149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:33.857788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:33.858067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:33.858371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:33.859537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] RBAC: Authorizing `identity:get_project(project_id=bcfe7298eddd4ed4acf5d3a34317af77)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:33.860973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:33.862681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 540/1066] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:33.868453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:33.907226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.957253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:33.965127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44bd22-da0b-45eb-92aa-af9d5b4f9d2d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:33.973494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcb74b-e1a5-4471-8bba-d566bb6b7323 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:34.003312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:34.003947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:34.004227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:34.004501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:34.010260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:34.010869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:34.011152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:34.011425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:34.012202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77, user.id=f8c655bdad6a4a70a131eb7884b223ee) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:34.012513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:34.014793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:34.015720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:34.017225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:34.020608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 541/1067] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/role_assignments?user.id=f8c655bdad6a4a70a131eb7884b223ee&scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:34.034789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:34.035782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:34.036919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:34.036919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:34.036919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:34.036919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:34.036919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 527/1068] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:34.042053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:34.152676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff438392-0eec-485f-9bfd-f4e9c584e436 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:34.188682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:34.189303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:34.189582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:34.189855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:34.190570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:34.190874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:34.192250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:34.206305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 542/1069] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:34.212079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:34.310833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:34.315719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61989cbf-3bca-4fd5-9a26-058301ef6428 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:34.350947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:34.352011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:34.352517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:34.353002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:34.360678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:34.362369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:34.375165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:34.376077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:34.376998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:34.383880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 528/1070] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:34.384980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:34.397381 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:34.491981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9fa0e-47ec-45d7-a146-fbff17769ffc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:34.526231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213f027-6bcf-4d75-86cd-4beb2efea4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:34.526850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f027-6bcf-4d75-86cd-4beb2efea4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:34.527131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f027-6bcf-4d75-86cd-4beb2efea4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:34.527406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f027-6bcf-4d75-86cd-4beb2efea4b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:34.528148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213f027-6bcf-4d75-86cd-4beb2efea4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:34.528477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213f027-6bcf-4d75-86cd-4beb2efea4b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:34.530418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213f027-6bcf-4d75-86cd-4beb2efea4b7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:34.547722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 543/1071] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:34.548986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:34.658465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781966-86c3-4081-9adb-333e3af5f91d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:34.692431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed19d5-af9f-4c2d-aa46-cacb5cf984db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:34.693095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed19d5-af9f-4c2d-aa46-cacb5cf984db None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:34.693376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed19d5-af9f-4c2d-aa46-cacb5cf984db None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:34.693651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed19d5-af9f-4c2d-aa46-cacb5cf984db None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e4e461b0b6142b38fd7de56395733b2/roles/7065360498844be996f65d53579e8ab3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:34.695878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed19d5-af9f-4c2d-aa46-cacb5cf984db None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=4e4e461b0b6142b38fd7de56395733b2, role_id=7065360498844be996f65d53579e8ab3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:34.697308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed19d5-af9f-4c2d-aa46-cacb5cf984db None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:34.802247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 529/1072] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:21:34 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/4e4e461b0b6142b38fd7de56395733b2/roles/7065360498844be996f65d53579e8ab3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:21:35.380680 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf7c-3bca-4499-9db6-a3bddc9a98b2 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:35.381058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf7c-3bca-4499-9db6-a3bddc9a98b2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:35.381233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0cf7c-3bca-4499-9db6-a3bddc9a98b2 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:35.382804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 544/1073] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:35.387702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a12621-6e3c-422e-b89e-89a00bb53fd1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:35.388150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a12621-6e3c-422e-b89e-89a00bb53fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:35.388338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a12621-6e3c-422e-b89e-89a00bb53fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:35.406585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-62a12621-6e3c-422e-b89e-89a00bb53fd1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:35.407769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-62a12621-6e3c-422e-b89e-89a00bb53fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:35.421603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:35.477283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 530/1074] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:35.499466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0cf7c-3bca-4499-9db6-a3bddc9a98b2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:35.576216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:35.610304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0cf7c-3bca-4499-9db6-a3bddc9a98b2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:35.645048 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:35.645591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:35.645932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:35.646235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:35.714923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:35.716459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:35.781205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:35.819972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 545/1075] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:35.821822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a12621-6e3c-422e-b89e-89a00bb53fd1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:35.924595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a12621-6e3c-422e-b89e-89a00bb53fd1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:35.959110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a01427-1e6c-4e3f-8a13-c33e12840c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:35.959744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a01427-1e6c-4e3f-8a13-c33e12840c8f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:35.960023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a01427-1e6c-4e3f-8a13-c33e12840c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:35.960294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a01427-1e6c-4e3f-8a13-c33e12840c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:35.970820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a01427-1e6c-4e3f-8a13-c33e12840c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:35.972251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a01427-1e6c-4e3f-8a13-c33e12840c8f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:35.980531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a8a01427-1e6c-4e3f-8a13-c33e12840c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:35.987859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:35.988974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:35.995058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:35.995058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:35.995058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:35.995058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:35.995058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 531/1076] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:35.996916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:36.025752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5d7dd-942b-438e-9587-7a254e6460b2 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.026341 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5d7dd-942b-438e-9587-7a254e6460b2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.026616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5d7dd-942b-438e-9587-7a254e6460b2 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.030316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 532/1077] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:21:36.033223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3aa85-a607-43ee-848e-0e0032d7b6b1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.033505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3aa85-a607-43ee-848e-0e0032d7b6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.033776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3aa85-a607-43ee-848e-0e0032d7b6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.051528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7ee3aa85-a607-43ee-848e-0e0032d7b6b1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:36.052688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7ee3aa85-a607-43ee-848e-0e0032d7b6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:36.107884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ded94-b6e7-4326-8bfe-1e08ddbdfa9c None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:36.127857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 533/1078] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:36.145026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:36.145674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.145967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.146263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.146950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:36.147257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:36.148720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:36.149580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3aa85-a607-43ee-848e-0e0032d7b6b1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:36.170170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 546/1079] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:36.266264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3aa85-a607-43ee-848e-0e0032d7b6b1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:36.302744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:36.309275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:36.310019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.310379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.310650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.321242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:36.326640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:36.327786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:36.328767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 534/1080] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:21:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:36.333892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:36.366370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:36.441526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3fd03-78c7-40e7-b986-c1b9c52b77f5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:36.475798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:36.476418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.476697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.476970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.477688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:36.477993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:36.479398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:36.488865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 547/1081] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:21:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:36.494859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:36.593037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305943e-80f4-4b5e-82a4-cdd909fcc89e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:36.626408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40c645-96a3-4e19-86d7-2194790c4470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:36.627037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40c645-96a3-4e19-86d7-2194790c4470 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.627314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40c645-96a3-4e19-86d7-2194790c4470 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.627588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40c645-96a3-4e19-86d7-2194790c4470 None admin] PATH_INFO: `/v3/users/f8c655bdad6a4a70a131eb7884b223ee` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.628736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40c645-96a3-4e19-86d7-2194790c4470 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c655bdad6a4a70a131eb7884b223ee)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:36.630167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40c645-96a3-4e19-86d7-2194790c4470 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:36.631956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 535/1082] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:36 2022] GET /identity/v3/users/f8c655bdad6a4a70a131eb7884b223ee => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:36.637439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:36.729461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:36.739984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291c50b-3020-4148-a693-ea052a54227e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:36.772710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:36.773338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.773617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.773892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.775087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] RBAC: Authorizing `identity:get_project(project_id=bcfe7298eddd4ed4acf5d3a34317af77)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:36.776518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:36.778202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 548/1083] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:21:36 2022] GET /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:36.784329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:36.784462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77965860-b303-4d81-a539-5779669113e1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.784749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77965860-b303-4d81-a539-5779669113e1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.785092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77965860-b303-4d81-a539-5779669113e1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.786544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 536/1084] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:36.791228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd1ab-b2e6-4102-886e-e729c028a53d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.791506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd1ab-b2e6-4102-886e-e729c028a53d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.791776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd1ab-b2e6-4102-886e-e729c028a53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.810361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5f8fd1ab-b2e6-4102-886e-e729c028a53d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:36.811550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5f8fd1ab-b2e6-4102-886e-e729c028a53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:36.863790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:36.877189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 537/1085] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 27 08:21:36.882312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aad9d4-0541-41fc-92a9-e7f9f921274d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:36.900299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd1ab-b2e6-4102-886e-e729c028a53d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:36.915470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:36.916131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:36.916410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:36.916681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77/users/f8c655bdad6a4a70a131eb7884b223ee/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:36.918640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] RBAC: Authorizing `identity:create_grant(project_id=bcfe7298eddd4ed4acf5d3a34317af77, user_id=f8c655bdad6a4a70a131eb7884b223ee, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:36.920917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:36.970577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 549/1086] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:21:36 2022] PUT /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77/users/f8c655bdad6a4a70a131eb7884b223ee/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:21:37.000620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd1ab-b2e6-4102-886e-e729c028a53d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:37.038258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:37.038916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:37.039195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:37.039464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:37.052127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:37.057155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:37.058414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:37.059289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 538/1087] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 27 08:21:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:21:37.064953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:37.084965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:37.170344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c2abe-faa5-4753-ae36-f2aa16cffd6a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:37.203745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:37.204374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:37.204658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:37.204933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:37.205650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:37.205958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:37.207360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:37.215885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 550/1088] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 27 08:21:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:37.221625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:37.271769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:37.322751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4719214-ce43-4097-a8e4-dc0245809012 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:37.357566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:37.358255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:37.358544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:37.358819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:37.368689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:37.370121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:37.387105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:37.388196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:37.389138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:37.389138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:37.389138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:37.389138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:37.389138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 539/1089] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:37.394371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:37.476451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:37.495126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d32ef1-6559-486b-b327-1b83ac1d1708 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:37.530372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:37.530984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:37.531267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:37.531543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:37.532233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:37.532541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:37.533904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:37.546960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 551/1090] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:37.552583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:37.648722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:37.651775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb37fc-6170-4b0c-8827-965ba5d64716 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:37.685232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:37.685857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:37.686132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:37.686429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:37.693806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:37.695256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:21:37.706132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:37.707133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:37.708081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 540/1091] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 27 08:21:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:37.713045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:37.809137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dddaaa-33d4-4c90-8292-4ec0fa7c5fa5 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:37.842324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:37.842919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:37.843194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:37.843463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:37.844237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:37.844792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:37.846227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:37.854581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:37.862762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 552/1092] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 27 08:21:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:37.868447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:37.968913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820bf1d-8390-4f2b-9a6c-a5cd6e42d9c4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:38.002164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:38.002772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.003051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.003323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.031905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:38.072670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:38.074041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:38.172930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 541/1093] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:38.178665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:38.277945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41707eb1-2452-4c60-bb3f-95efbc9adc25 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:38.310956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:38.311582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.311864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.312138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.321841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:38.323375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:21:38.338028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:38.339107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:38.340043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:21:38.340043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:21:38.340043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:21:38.340043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:38.340043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 553/1094] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:38.348038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:38.406320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:38.446308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be43ca-1e26-4370-93b1-80462f04a85e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:38.479719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:38.480324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.480605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.480877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.482467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:38.482788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:38.484145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:38.495103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 542/1095] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:38.500739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:38.552514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:38.604351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e4b6e-42b9-46bc-9a6c-8f5e68c686e6 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:38.636832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:38.637464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.638337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.638772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.645331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:38.646812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:21:38.658579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:38.659621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:21:38.660531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 554/1096] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:38.665549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:38.761524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60435049-1402-416a-8ea9-43075a5b3667 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:38.781426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:38.794698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6050b55-1768-4069-862b-cac28ab35423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:38.795302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6050b55-1768-4069-862b-cac28ab35423 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.795577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6050b55-1768-4069-862b-cac28ab35423 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.795844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6050b55-1768-4069-862b-cac28ab35423 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.796581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6050b55-1768-4069-862b-cac28ab35423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:38.796883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6050b55-1768-4069-862b-cac28ab35423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:38.798307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6050b55-1768-4069-862b-cac28ab35423 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:38.810085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 543/1097] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:38.816169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:38.917614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd52198-fbea-443c-969c-64133171c13d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:38.928121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:38.951775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833860e-b667-4a29-95bd-790d401dbe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:38.952383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833860e-b667-4a29-95bd-790d401dbe19 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.956337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833860e-b667-4a29-95bd-790d401dbe19 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.956337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833860e-b667-4a29-95bd-790d401dbe19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.956337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833860e-b667-4a29-95bd-790d401dbe19 None admin] RBAC: Adding query filter params (role.id=7065360498844be996f65d53579e8ab3, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=4e4e461b0b6142b38fd7de56395733b2) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:38.956337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833860e-b667-4a29-95bd-790d401dbe19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:38.956337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833860e-b667-4a29-95bd-790d401dbe19 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:38.960173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ff506-0175-4c64-b74c-a44fe3b950d6 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.960560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ff506-0175-4c64-b74c-a44fe3b950d6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.960839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ff506-0175-4c64-b74c-a44fe3b950d6 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.970984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 555/1098] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:38 2022] GET /identity/v3/role_assignments?user.id=4e4e461b0b6142b38fd7de56395733b2&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=7065360498844be996f65d53579e8ab3 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:21:38.971203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 544/1099] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:38.971330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa1eb-a0ca-4d21-98f9-76b62d2c218e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:38.971330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa1eb-a0ca-4d21-98f9-76b62d2c218e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:38.971330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa1eb-a0ca-4d21-98f9-76b62d2c218e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:38.989085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d64aa1eb-a0ca-4d21-98f9-76b62d2c218e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:38.990361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d64aa1eb-a0ca-4d21-98f9-76b62d2c218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:39.062800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 556/1100] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:39.086971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ff506-0175-4c64-b74c-a44fe3b950d6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:39.190864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ff506-0175-4c64-b74c-a44fe3b950d6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:39.224971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:39.226170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:39.226451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:39.226711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:39.238274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:39.243177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:39.244340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:39.245239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 545/1101] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:39.250947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa1eb-a0ca-4d21-98f9-76b62d2c218e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:39.351532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa1eb-a0ca-4d21-98f9-76b62d2c218e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:39.353876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:39.389631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:39.390469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:39.391015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:39.391285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:39.392004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:39.392301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:39.393669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:39.402364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 557/1102] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:39.409135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:39.452388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:39.513120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b811410-9e87-4d16-824e-11d26ecc0f02 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:39.548462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:39.549110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:39.549390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:39.549671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] PATH_INFO: `/v3/users/f8c655bdad6a4a70a131eb7884b223ee` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:39.550890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] RBAC: Authorizing `identity:get_user(user_id=f8c655bdad6a4a70a131eb7884b223ee)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:39.552325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:39.553939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 546/1103] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:39 2022] GET /identity/v3/users/f8c655bdad6a4a70a131eb7884b223ee => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:39.560067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:39.668059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70f71c-1443-48f5-a60f-2d9b2de9ca9d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:39.703361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:39.703909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:39.704188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:39.704461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:39.737889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:39.771529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:39.773728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:39.870787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 558/1104] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:39.876632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:39.972103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3438f57-9221-45c4-9dc3-5c3396958943 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:39.980794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:40.004624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287b5d2-10bd-49e8-bfc5-0f76acc50d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:40.005243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287b5d2-10bd-49e8-bfc5-0f76acc50d92 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:40.005521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287b5d2-10bd-49e8-bfc5-0f76acc50d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:40.005790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287b5d2-10bd-49e8-bfc5-0f76acc50d92 None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:40.006957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287b5d2-10bd-49e8-bfc5-0f76acc50d92 None admin] RBAC: Authorizing `identity:get_project(project_id=bcfe7298eddd4ed4acf5d3a34317af77)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:40.008365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287b5d2-10bd-49e8-bfc5-0f76acc50d92 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:40.009979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 547/1105] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:21:39 2022] GET /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:40.015997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:40.078960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:40.112084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be268379-fd1a-413e-bc88-54e78efd8d00 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:40.144484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab431d-02ce-46e7-84de-c23a314ee8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:40.145081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab431d-02ce-46e7-84de-c23a314ee8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:40.145359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab431d-02ce-46e7-84de-c23a314ee8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:40.145634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab431d-02ce-46e7-84de-c23a314ee8a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:40.146441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab431d-02ce-46e7-84de-c23a314ee8a7 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77, user.id=f8c655bdad6a4a70a131eb7884b223ee) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:40.146750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab431d-02ce-46e7-84de-c23a314ee8a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:40.148169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab431d-02ce-46e7-84de-c23a314ee8a7 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:40.154637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 559/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:40 2022] GET /identity/v3/role_assignments?user.id=f8c655bdad6a4a70a131eb7884b223ee&scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:42.162355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02ad2-0fdd-447a-ac7d-64422db50524 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:42.162924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02ad2-0fdd-447a-ac7d-64422db50524 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:42.162924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02ad2-0fdd-447a-ac7d-64422db50524 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:42.164258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 548/1107] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:42.169346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058212ec-21d5-4e32-9908-17205df54693 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:42.169737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058212ec-21d5-4e32-9908-17205df54693 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:42.169912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058212ec-21d5-4e32-9908-17205df54693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:42.188019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-058212ec-21d5-4e32-9908-17205df54693 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:42.189201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-058212ec-21d5-4e32-9908-17205df54693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:42.254904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 560/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:42.279653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02ad2-0fdd-447a-ac7d-64422db50524 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:42.377416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02ad2-0fdd-447a-ac7d-64422db50524 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:42.409164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed20bb4-b33a-427f-bbab-9db0b8058e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:42.409775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20bb4-b33a-427f-bbab-9db0b8058e34 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:42.410050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20bb4-b33a-427f-bbab-9db0b8058e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:42.410348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20bb4-b33a-427f-bbab-9db0b8058e34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:42.411469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed20bb4-b33a-427f-bbab-9db0b8058e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:42.412919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed20bb4-b33a-427f-bbab-9db0b8058e34 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:42.414432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 549/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:42.420583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058212ec-21d5-4e32-9908-17205df54693 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:42.476919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:42.515037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058212ec-21d5-4e32-9908-17205df54693 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:42.548501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c1f3a-0b40-45c3-9b4a-5d13c616a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:42.549105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1f3a-0b40-45c3-9b4a-5d13c616a0e0 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:42.549382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1f3a-0b40-45c3-9b4a-5d13c616a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:42.549651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1f3a-0b40-45c3-9b4a-5d13c616a0e0 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:42.550344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c1f3a-0b40-45c3-9b4a-5d13c616a0e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:42.551772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c1f3a-0b40-45c3-9b4a-5d13c616a0e0 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:42.648988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 561/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:21:42 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:44.696144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4c72-a246-4034-a53f-6ba12da4e308 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:44.696693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4c72-a246-4034-a53f-6ba12da4e308 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:44.696693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4c72-a246-4034-a53f-6ba12da4e308 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:44.698016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 550/1111] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:44.703174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd02ddb-b0ea-4a40-a92c-cdc81c7cfbaf None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:44.703551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd02ddb-b0ea-4a40-a92c-cdc81c7cfbaf None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:44.703723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd02ddb-b0ea-4a40-a92c-cdc81c7cfbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:44.721576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fbd02ddb-b0ea-4a40-a92c-cdc81c7cfbaf None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:44.722794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fbd02ddb-b0ea-4a40-a92c-cdc81c7cfbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:44.795938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 562/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:44.818662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e4c72-a246-4034-a53f-6ba12da4e308 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:44.864725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:44.917076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e4c72-a246-4034-a53f-6ba12da4e308 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:44.949489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:44.950771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:44.951046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:44.951309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] PATH_INFO: `/v3/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:44.952727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] RBAC: Authorizing `identity:get_role(role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:44.954068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:44.955397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 551/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:44 2022] GET /identity/v3/roles/d469c6f0fa524f36949e61ae51807884 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:44.961265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd02ddb-b0ea-4a40-a92c-cdc81c7cfbaf None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:45.006077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:45.056917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd02ddb-b0ea-4a40-a92c-cdc81c7cfbaf None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:45.090937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:45.091555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:45.091835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:45.092110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] PATH_INFO: `/v3/users/7687186deb984da2bca0ed5118b98dd0` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:45.104857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] RBAC: Authorizing `identity:get_user(user_id=7687186deb984da2bca0ed5118b98dd0)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:45.106345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:45.107951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 563/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:44 2022] GET /identity/v3/users/7687186deb984da2bca0ed5118b98dd0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:45.113549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:45.208753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054bb5f-d962-4616-97a5-0a95a4e2e69a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:45.222159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:45.241955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:45.242581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:45.242855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:45.243120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:45.307281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:45.308703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:45.401953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 552/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:45.408576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:45.509259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64baf9ae-682a-4c35-9431-7d579dbf4147 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:45.511431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:45.543008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e61eb-714a-42d6-9a54-fed847a16a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:45.543884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e61eb-714a-42d6-9a54-fed847a16a39 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:45.544240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e61eb-714a-42d6-9a54-fed847a16a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:45.544516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e61eb-714a-42d6-9a54-fed847a16a39 None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:45.545815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e61eb-714a-42d6-9a54-fed847a16a39 None admin] RBAC: Authorizing `identity:get_project(project_id=bcfe7298eddd4ed4acf5d3a34317af77)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:45.547266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e61eb-714a-42d6-9a54-fed847a16a39 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:45.548989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 564/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:21:45 2022] GET /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:45.555545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:45.570110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:45.669722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36371715-9218-4d63-a2a0-efdb3d134d50 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:45.709026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e487430-3219-404c-824d-2694fa5c84eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:45.709705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487430-3219-404c-824d-2694fa5c84eb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:45.709908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487430-3219-404c-824d-2694fa5c84eb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:45.710208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e487430-3219-404c-824d-2694fa5c84eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:45.710999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e487430-3219-404c-824d-2694fa5c84eb None admin] RBAC: Adding query filter params (role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77, user.id=7687186deb984da2bca0ed5118b98dd0) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:45.711307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e487430-3219-404c-824d-2694fa5c84eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:45.712734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e487430-3219-404c-824d-2694fa5c84eb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:45.719892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 553/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:45 2022] GET /identity/v3/role_assignments?user.id=7687186deb984da2bca0ed5118b98dd0&scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77&role.id=d469c6f0fa524f36949e61ae51807884 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:47.700629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab5c13-bcbe-41a4-9ac9-3366d485503b None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:47.701184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab5c13-bcbe-41a4-9ac9-3366d485503b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:47.701184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab5c13-bcbe-41a4-9ac9-3366d485503b None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:47.702691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 565/1118] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:47.707722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e129-d27c-4763-a0fd-bb80d7748f5e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:47.708167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e129-d27c-4763-a0fd-bb80d7748f5e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:47.708270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e129-d27c-4763-a0fd-bb80d7748f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:47.726249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5c44e129-d27c-4763-a0fd-bb80d7748f5e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:47.727416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5c44e129-d27c-4763-a0fd-bb80d7748f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:47.792017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 554/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:47.813751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab5c13-bcbe-41a4-9ac9-3366d485503b None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:47.913843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab5c13-bcbe-41a4-9ac9-3366d485503b None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:47.946851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:47.947483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:47.947762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:47.948036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] PATH_INFO: `/v3/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:47.950036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] RBAC: Authorizing `identity:get_role(role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:47.951508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:47.952823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 566/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:47 2022] GET /identity/v3/roles/d469c6f0fa524f36949e61ae51807884 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:47.958502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44e129-d27c-4763-a0fd-bb80d7748f5e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:47.966387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:48.055237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44e129-d27c-4763-a0fd-bb80d7748f5e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:48.087969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:48.088613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:48.088887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:48.089157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] PATH_INFO: `/v3/users/7687186deb984da2bca0ed5118b98dd0` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:48.090489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] RBAC: Authorizing `identity:get_user(user_id=7687186deb984da2bca0ed5118b98dd0)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:48.091901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:48.093467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 555/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:47 2022] GET /identity/v3/users/7687186deb984da2bca0ed5118b98dd0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:48.099338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:48.141091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:48.195377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594bdba-8f4a-457d-be91-3b385595e33a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:48.227821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcb2cc-1ae6-4614-afe3-4a6bab5a5e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:48.228427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb2cc-1ae6-4614-afe3-4a6bab5a5e82 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:48.228700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb2cc-1ae6-4614-afe3-4a6bab5a5e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:48.228964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcb2cc-1ae6-4614-afe3-4a6bab5a5e82 None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:48.230079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb2cc-1ae6-4614-afe3-4a6bab5a5e82 None admin] RBAC: Authorizing `identity:get_project(project_id=bcfe7298eddd4ed4acf5d3a34317af77)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:48.231840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcb2cc-1ae6-4614-afe3-4a6bab5a5e82 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:48.233461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 567/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:21:48 2022] GET /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:48.239299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:48.311131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:48.334624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9e4e-f16c-4db0-bfa5-70d8847268df None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:48.368141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b4de3-0302-41f5-ae14-9d8f0d0864cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:48.368782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b4de3-0302-41f5-ae14-9d8f0d0864cc None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:48.369058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b4de3-0302-41f5-ae14-9d8f0d0864cc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:48.369326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b4de3-0302-41f5-ae14-9d8f0d0864cc None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77/users/7687186deb984da2bca0ed5118b98dd0/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:48.372166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b4de3-0302-41f5-ae14-9d8f0d0864cc None admin] RBAC: Authorizing `identity:create_grant(project_id=bcfe7298eddd4ed4acf5d3a34317af77, user_id=7687186deb984da2bca0ed5118b98dd0, role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:48.373690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b4de3-0302-41f5-ae14-9d8f0d0864cc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:48.437409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 556/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:21:48 2022] PUT /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77/users/7687186deb984da2bca0ed5118b98dd0/roles/d469c6f0fa524f36949e61ae51807884 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:21:50.389280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c254fe-4fa4-4391-95b4-d8747eb64159 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:50.389840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c254fe-4fa4-4391-95b4-d8747eb64159 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:50.389840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c254fe-4fa4-4391-95b4-d8747eb64159 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:50.391232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 568/1124] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:50.396424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ae1ff-42ff-49a3-b360-80364e5ef6f3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:50.396838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ae1ff-42ff-49a3-b360-80364e5ef6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:50.396970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ae1ff-42ff-49a3-b360-80364e5ef6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:50.416146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-de9ae1ff-42ff-49a3-b360-80364e5ef6f3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:50.417329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-de9ae1ff-42ff-49a3-b360-80364e5ef6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:50.480851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 557/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:50.504205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c254fe-4fa4-4391-95b4-d8747eb64159 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:50.603106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c254fe-4fa4-4391-95b4-d8747eb64159 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:50.612422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:50.636151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:50.637843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:50.637843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:50.637843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] PATH_INFO: `/v3/roles/d469c6f0fa524f36949e61ae51807884` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:50.639004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] RBAC: Authorizing `identity:get_role(role_id=d469c6f0fa524f36949e61ae51807884)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:50.640409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:50.641684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 569/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:50 2022] GET /identity/v3/roles/d469c6f0fa524f36949e61ae51807884 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:50.647526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ae1ff-42ff-49a3-b360-80364e5ef6f3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:50.706724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:50.742955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ae1ff-42ff-49a3-b360-80364e5ef6f3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:50.775754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:50.776373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:50.776648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:50.776916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] PATH_INFO: `/v3/users/7687186deb984da2bca0ed5118b98dd0` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:50.778044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] RBAC: Authorizing `identity:get_user(user_id=7687186deb984da2bca0ed5118b98dd0)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:50.779561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:50.781123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 558/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:50 2022] GET /identity/v3/users/7687186deb984da2bca0ed5118b98dd0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:50.786804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:50.883339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b27f5-19b0-4165-b157-571d115e4c9b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:50.915846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:50.916380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:50.916654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:50.916922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:50.959602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:50.981528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:50.983835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:51.076155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 570/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:51.082255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:51.178779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82276cc6-2473-44c8-8fa8-f36a2bb321d1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:51.206677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:51.211374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f5133-4295-4dcb-a535-3684e0571277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:51.211997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5133-4295-4dcb-a535-3684e0571277 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:51.212298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5133-4295-4dcb-a535-3684e0571277 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:51.212573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5133-4295-4dcb-a535-3684e0571277 None admin] PATH_INFO: `/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:51.213721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f5133-4295-4dcb-a535-3684e0571277 None admin] RBAC: Authorizing `identity:get_project(project_id=bcfe7298eddd4ed4acf5d3a34317af77)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:51.215165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f5133-4295-4dcb-a535-3684e0571277 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:51.218581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 559/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:21:51 2022] GET /identity/v3/projects/bcfe7298eddd4ed4acf5d3a34317af77 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:51.223779 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:51.297433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:51.318923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6413e97-638d-4e96-b507-3c8e2be165a0 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:51.352210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4c2ec-c88b-4766-b77f-a8cc1a19e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:51.352806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4c2ec-c88b-4766-b77f-a8cc1a19e9ba None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:51.353082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4c2ec-c88b-4766-b77f-a8cc1a19e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:51.353353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4c2ec-c88b-4766-b77f-a8cc1a19e9ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:51.354121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4c2ec-c88b-4766-b77f-a8cc1a19e9ba None admin] RBAC: Adding query filter params (role.id=d469c6f0fa524f36949e61ae51807884, scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77, user.id=7687186deb984da2bca0ed5118b98dd0) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:51.354457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4c2ec-c88b-4766-b77f-a8cc1a19e9ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:51.355869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4c2ec-c88b-4766-b77f-a8cc1a19e9ba None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:51.361537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 571/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:51 2022] GET /identity/v3/role_assignments?user.id=7687186deb984da2bca0ed5118b98dd0&scope.project.id=bcfe7298eddd4ed4acf5d3a34317af77&role.id=d469c6f0fa524f36949e61ae51807884 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:53.407584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05986f94-1f27-44c8-b94d-515f23b2c924 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:53.408141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05986f94-1f27-44c8-b94d-515f23b2c924 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:53.408141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05986f94-1f27-44c8-b94d-515f23b2c924 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:53.409495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 560/1131] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:53.414643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47361fa-d5c4-462d-8c61-88c8140541fe None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:53.415045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47361fa-d5c4-462d-8c61-88c8140541fe None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:53.415188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47361fa-d5c4-462d-8c61-88c8140541fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:53.433331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f47361fa-d5c4-462d-8c61-88c8140541fe None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:53.434543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f47361fa-d5c4-462d-8c61-88c8140541fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:53.500758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 572/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:53.524080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05986f94-1f27-44c8-b94d-515f23b2c924 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:53.623813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05986f94-1f27-44c8-b94d-515f23b2c924 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:53.656750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfacdfc-92d8-4329-b70a-da7ce5bb6c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:53.657369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfacdfc-92d8-4329-b70a-da7ce5bb6c36 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:53.657699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfacdfc-92d8-4329-b70a-da7ce5bb6c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:53.657921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfacdfc-92d8-4329-b70a-da7ce5bb6c36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:53.659086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfacdfc-92d8-4329-b70a-da7ce5bb6c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:53.660500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfacdfc-92d8-4329-b70a-da7ce5bb6c36 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:53.662063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 561/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:53.667921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47361fa-d5c4-462d-8c61-88c8140541fe None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:53.734183 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:53.762619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47361fa-d5c4-462d-8c61-88c8140541fe None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:53.795962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265364c2-1382-4846-b79a-8a5aed43357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:53.796552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265364c2-1382-4846-b79a-8a5aed43357c None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:53.796826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265364c2-1382-4846-b79a-8a5aed43357c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:53.797093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265364c2-1382-4846-b79a-8a5aed43357c None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:53.797768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265364c2-1382-4846-b79a-8a5aed43357c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:53.799831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265364c2-1382-4846-b79a-8a5aed43357c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:53.878386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 573/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:21:53 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:55.886940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2cf75-df3d-4a2a-b22c-4c32624315da None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:55.887481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2cf75-df3d-4a2a-b22c-4c32624315da None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:55.887481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2cf75-df3d-4a2a-b22c-4c32624315da None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:55.889311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 562/1135] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:55.894498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7472fe-03ae-4160-ad6a-f03081c4915e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:55.894884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7472fe-03ae-4160-ad6a-f03081c4915e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:55.895047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7472fe-03ae-4160-ad6a-f03081c4915e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:55.913312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4e7472fe-03ae-4160-ad6a-f03081c4915e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:55.914514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4e7472fe-03ae-4160-ad6a-f03081c4915e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:55.978663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 574/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:56.003531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2cf75-df3d-4a2a-b22c-4c32624315da None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:56.074370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:56.108239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2cf75-df3d-4a2a-b22c-4c32624315da None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:56.141732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daa7d9-8670-4371-a055-7d1525d01643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:56.142323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daa7d9-8670-4371-a055-7d1525d01643 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:56.142602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daa7d9-8670-4371-a055-7d1525d01643 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:56.142873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daa7d9-8670-4371-a055-7d1525d01643 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:56.143995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daa7d9-8670-4371-a055-7d1525d01643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:56.145409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daa7d9-8670-4371-a055-7d1525d01643 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:56.146989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 563/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:21:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:56.152754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7472fe-03ae-4160-ad6a-f03081c4915e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:56.203211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:56.249194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7472fe-03ae-4160-ad6a-f03081c4915e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:56.282647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea5398-f0a5-4909-a075-d540f38b215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:56.283251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea5398-f0a5-4909-a075-d540f38b215b None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:56.283530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea5398-f0a5-4909-a075-d540f38b215b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:56.283802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea5398-f0a5-4909-a075-d540f38b215b None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:56.284480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea5398-f0a5-4909-a075-d540f38b215b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:56.285904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea5398-f0a5-4909-a075-d540f38b215b None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:56.385685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 575/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:21:56 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:21:58.429716 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc882af-cd0d-46e7-8317-b3c2e71d3376 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:58.430301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc882af-cd0d-46e7-8317-b3c2e71d3376 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:58.430301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc882af-cd0d-46e7-8317-b3c2e71d3376 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:58.431618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 564/1139] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:21:58.437181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d802dc6-ee67-4c11-aa6c-971d133b3c37 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:58.437604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d802dc6-ee67-4c11-aa6c-971d133b3c37 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:58.437703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d802dc6-ee67-4c11-aa6c-971d133b3c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:58.455855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2d802dc6-ee67-4c11-aa6c-971d133b3c37 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:21:58.457020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2d802dc6-ee67-4c11-aa6c-971d133b3c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:21:58.521050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 576/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:21:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:21:58.543717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc882af-cd0d-46e7-8317-b3c2e71d3376 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:58.642350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc882af-cd0d-46e7-8317-b3c2e71d3376 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:58.674332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:58.674971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:58.675247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:58.675511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:58.678037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:58.686091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:21:58.690881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:21:58.692075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:21:58.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 565/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:21:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:21:58.698163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d802dc6-ee67-4c11-aa6c-971d133b3c37 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:58.737231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:58.793808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d802dc6-ee67-4c11-aa6c-971d133b3c37 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:58.827636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:58.828239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:58.828523 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:58.828797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:58.829510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:58.829813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:58.831229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:58.839057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 577/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:21:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:58.845105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:58.943831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21c535-d891-496b-bc5b-e127a9092847 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:58.975908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:58.976529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:58.976808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:58.977080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] PATH_INFO: `/v3/users/97007f0e3bc84704b2811752d920eef4` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:58.990089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] RBAC: Authorizing `identity:get_user(user_id=97007f0e3bc84704b2811752d920eef4)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:58.991545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:58.993130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 566/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:21:58 2022] GET /identity/v3/users/97007f0e3bc84704b2811752d920eef4 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:58.999550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:59.093891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:59.096952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539cc76-4302-4309-a80f-f1fddd17d743 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:59.128823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:59.129421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:59.129697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:59.129966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:59.195607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:59.196980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:59.287630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:59.291112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 578/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:21:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:21:59.297638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:59.351183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:21:59.393862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4aacb-bdea-42e4-a31b-6ff744cf9453 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:59.427212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9884651-2622-4a63-beda-d683b5897e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:59.427828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9884651-2622-4a63-beda-d683b5897e97 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:59.428107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9884651-2622-4a63-beda-d683b5897e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:59.428378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9884651-2622-4a63-beda-d683b5897e97 None admin] PATH_INFO: `/v3/projects/d1ef7359dbf848eb87334f43225a6f77` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:59.429506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9884651-2622-4a63-beda-d683b5897e97 None admin] RBAC: Authorizing `identity:get_project(project_id=d1ef7359dbf848eb87334f43225a6f77)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:59.430945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9884651-2622-4a63-beda-d683b5897e97 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:59.432561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 567/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:21:59 2022] GET /identity/v3/projects/d1ef7359dbf848eb87334f43225a6f77 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:21:59.438695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:21:59.534424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b3d7-7389-4f5c-af8b-ed0158d550be None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:21:59.566803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ce0b1-9712-4f85-b84a-357bc064095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:21:59.567412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce0b1-9712-4f85-b84a-357bc064095f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:21:59.567691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce0b1-9712-4f85-b84a-357bc064095f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:21:59.567962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce0b1-9712-4f85-b84a-357bc064095f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:21:59.569118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ce0b1-9712-4f85-b84a-357bc064095f None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=d1ef7359dbf848eb87334f43225a6f77, user.id=97007f0e3bc84704b2811752d920eef4) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:21:59.569118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ce0b1-9712-4f85-b84a-357bc064095f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:21:59.570479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ce0b1-9712-4f85-b84a-357bc064095f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:21:59.575768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 579/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:21:59 2022] GET /identity/v3/role_assignments?user.id=97007f0e3bc84704b2811752d920eef4&scope.project.id=d1ef7359dbf848eb87334f43225a6f77&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:01.537925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bb47e-fe06-4cd5-beda-768f866865eb None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:01.538512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bb47e-fe06-4cd5-beda-768f866865eb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:01.538512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bb47e-fe06-4cd5-beda-768f866865eb None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:01.539836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 568/1147] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:01.545487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab97575-e220-4945-b20a-d2e49ddeb39e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:01.545861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab97575-e220-4945-b20a-d2e49ddeb39e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:01.546116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab97575-e220-4945-b20a-d2e49ddeb39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:01.564547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fab97575-e220-4945-b20a-d2e49ddeb39e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:01.565730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fab97575-e220-4945-b20a-d2e49ddeb39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:01.623879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:01.631992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 580/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:01.654429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bb47e-fe06-4cd5-beda-768f866865eb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:01.751432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bb47e-fe06-4cd5-beda-768f866865eb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:01.773171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:01.784667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:01.785287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:01.785560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:01.785828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:01.795664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:22:01.800805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:22:01.801921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:22:01.802895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 569/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:22:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:22:01.808321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab97575-e220-4945-b20a-d2e49ddeb39e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:01.905735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab97575-e220-4945-b20a-d2e49ddeb39e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:01.937771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:01.938396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:01.938676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:01.938948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:01.939651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:01.939953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:01.941315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:01.949054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 581/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:22:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:01.955283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:02.052480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89684a-4b67-4aaf-b800-6b040a42cf91 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:02.085627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:02.086271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:02.086553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:02.086825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] PATH_INFO: `/v3/users/97007f0e3bc84704b2811752d920eef4` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:02.087971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=97007f0e3bc84704b2811752d920eef4)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:02.089377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:02.090967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 570/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:22:01 2022] GET /identity/v3/users/97007f0e3bc84704b2811752d920eef4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:02.097366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:02.152009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:02.194642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adc627-c2c4-4071-aba3-fa5c7924e1fb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:02.227234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3f026-c0d5-4f0a-abc1-a1f2c355ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:02.227857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3f026-c0d5-4f0a-abc1-a1f2c355ca70 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:02.228164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3f026-c0d5-4f0a-abc1-a1f2c355ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:02.228435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3f026-c0d5-4f0a-abc1-a1f2c355ca70 None admin] PATH_INFO: `/v3/projects/d1ef7359dbf848eb87334f43225a6f77` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:02.229564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3f026-c0d5-4f0a-abc1-a1f2c355ca70 None admin] RBAC: Authorizing `identity:get_project(project_id=d1ef7359dbf848eb87334f43225a6f77)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:02.231016 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3f026-c0d5-4f0a-abc1-a1f2c355ca70 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:02.232815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 582/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:22:02 2022] GET /identity/v3/projects/d1ef7359dbf848eb87334f43225a6f77 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:02.238697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:02.287790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:02.336118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9d35d-d497-4eac-95e5-f91b106dd0c1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:02.369128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c734b99-c0ed-4c55-9d09-f120fb3a655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:02.369779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734b99-c0ed-4c55-9d09-f120fb3a655f None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:02.370056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734b99-c0ed-4c55-9d09-f120fb3a655f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:02.370353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c734b99-c0ed-4c55-9d09-f120fb3a655f None admin] PATH_INFO: `/v3/projects/d1ef7359dbf848eb87334f43225a6f77/users/97007f0e3bc84704b2811752d920eef4/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:02.372234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c734b99-c0ed-4c55-9d09-f120fb3a655f None admin] RBAC: Authorizing `identity:create_grant(project_id=d1ef7359dbf848eb87334f43225a6f77, user_id=97007f0e3bc84704b2811752d920eef4, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:02.374277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c734b99-c0ed-4c55-9d09-f120fb3a655f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:02.457210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 571/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:22:02 2022] PUT /identity/v3/projects/d1ef7359dbf848eb87334f43225a6f77/users/97007f0e3bc84704b2811752d920eef4/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:22:04.432736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197788a6-7c18-4f0b-8e43-af0e73a47e7f None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:04.433288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197788a6-7c18-4f0b-8e43-af0e73a47e7f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:04.433288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197788a6-7c18-4f0b-8e43-af0e73a47e7f None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:04.434670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 583/1154] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:04.440361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7b869-7f09-4088-85cb-7e2cfbcf51c3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:04.440780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7b869-7f09-4088-85cb-7e2cfbcf51c3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:04.440912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7b869-7f09-4088-85cb-7e2cfbcf51c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:04.459109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c2d7b869-7f09-4088-85cb-7e2cfbcf51c3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:04.460289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c2d7b869-7f09-4088-85cb-7e2cfbcf51c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:04.524884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 572/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:04.549287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197788a6-7c18-4f0b-8e43-af0e73a47e7f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:04.648244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197788a6-7c18-4f0b-8e43-af0e73a47e7f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:04.670828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:04.680959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:04.681582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:04.681860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:04.692279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:04.692856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:22:04.697610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:22:04.698708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:22:04.699878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 584/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:22:04.705517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7b869-7f09-4088-85cb-7e2cfbcf51c3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:04.745180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:04.801662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7b869-7f09-4088-85cb-7e2cfbcf51c3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:04.834940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:04.835541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:04.835820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:04.836087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:04.836786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:04.837085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:04.838456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:04.847175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 573/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:22:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:04.852880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:04.946690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280ab82-be06-4697-bab1-14ecb463139a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:04.979152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:04.979763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:04.980038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:04.980315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] PATH_INFO: `/v3/users/97007f0e3bc84704b2811752d920eef4` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:04.981435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] RBAC: Authorizing `identity:get_user(user_id=97007f0e3bc84704b2811752d920eef4)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:04.982914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:04.984464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 585/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:22:04 2022] GET /identity/v3/users/97007f0e3bc84704b2811752d920eef4 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:04.990079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:05.085978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831cf9-a534-4210-8c16-d9e6538f4e0a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:05.087501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:05.119959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:05.120561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:05.120843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:05.121113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:05.185485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:05.186936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:05.280904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 574/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:22:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:22:05.286659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:05.338549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:05.384803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005a329-ddcb-4754-b4ae-a7b09aa7c846 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:05.429804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af99808-cbc9-4759-be92-c1a8d7e90740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:05.430457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af99808-cbc9-4759-be92-c1a8d7e90740 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:05.430739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af99808-cbc9-4759-be92-c1a8d7e90740 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:05.431015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af99808-cbc9-4759-be92-c1a8d7e90740 None admin] PATH_INFO: `/v3/projects/d1ef7359dbf848eb87334f43225a6f77` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:05.432155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af99808-cbc9-4759-be92-c1a8d7e90740 None admin] RBAC: Authorizing `identity:get_project(project_id=d1ef7359dbf848eb87334f43225a6f77)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:05.433593 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af99808-cbc9-4759-be92-c1a8d7e90740 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:05.435874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 586/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:22:05 2022] GET /identity/v3/projects/d1ef7359dbf848eb87334f43225a6f77 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:05.441884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:05.444052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:05.537814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ea342-0f5d-4ba6-9f8b-68c5563a2db9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:05.570268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b0426-399e-475f-820b-bb66dd040460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:05.570879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b0426-399e-475f-820b-bb66dd040460 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:05.571156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b0426-399e-475f-820b-bb66dd040460 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:05.571424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b0426-399e-475f-820b-bb66dd040460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:05.572238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b0426-399e-475f-820b-bb66dd040460 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=d1ef7359dbf848eb87334f43225a6f77, user.id=97007f0e3bc84704b2811752d920eef4) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:05.572546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b0426-399e-475f-820b-bb66dd040460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:05.573957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b0426-399e-475f-820b-bb66dd040460 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:05.580159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 575/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:22:05 2022] GET /identity/v3/role_assignments?user.id=97007f0e3bc84704b2811752d920eef4&scope.project.id=d1ef7359dbf848eb87334f43225a6f77&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:07.621315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35727406-071e-4b85-b894-908e3623abb5 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:07.621819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35727406-071e-4b85-b894-908e3623abb5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:07.621920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35727406-071e-4b85-b894-908e3623abb5 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:07.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 587/1162] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:07.628319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafdc0d-22be-4fb3-8446-34e30e4e2c6b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:07.628748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafdc0d-22be-4fb3-8446-34e30e4e2c6b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:07.628850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafdc0d-22be-4fb3-8446-34e30e4e2c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:07.647073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7fafdc0d-22be-4fb3-8446-34e30e4e2c6b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:07.648252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7fafdc0d-22be-4fb3-8446-34e30e4e2c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:07.706377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:07.713381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 576/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:07.736299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35727406-071e-4b85-b894-908e3623abb5 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:07.834912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35727406-071e-4b85-b894-908e3623abb5 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:07.853859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:07.867565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:07.868169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:07.868448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:07.868722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:07.933370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:07.934871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:08.028991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 588/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:22:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:22:08.035282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafdc0d-22be-4fb3-8446-34e30e4e2c6b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:08.131637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafdc0d-22be-4fb3-8446-34e30e4e2c6b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:08.164494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753442-2454-4950-8271-3b0b768969df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:08.165112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753442-2454-4950-8271-3b0b768969df None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:08.165387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753442-2454-4950-8271-3b0b768969df None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:08.165652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753442-2454-4950-8271-3b0b768969df None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:08.172228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b753442-2454-4950-8271-3b0b768969df None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:08.173635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b753442-2454-4950-8271-3b0b768969df None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4b753442-2454-4950-8271-3b0b768969df None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:22:08.184591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:22:08.186442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:22:08.187513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 577/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:22:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:22:08.192136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:08.213053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:08.289198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bfda8-05bf-4951-b7ea-c5739ccbcb51 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:08.322003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04752b93-8988-4fc9-a5ba-7c1dc4094a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:08.322630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04752b93-8988-4fc9-a5ba-7c1dc4094a94 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:08.322908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04752b93-8988-4fc9-a5ba-7c1dc4094a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:08.323176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04752b93-8988-4fc9-a5ba-7c1dc4094a94 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:08.323858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04752b93-8988-4fc9-a5ba-7c1dc4094a94 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:08.324160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04752b93-8988-4fc9-a5ba-7c1dc4094a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:08.325516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04752b93-8988-4fc9-a5ba-7c1dc4094a94 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:08.332998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 589/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 27 08:22:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:10.297823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7183f-a7bf-430b-b985-11df611c9c99 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:10.298381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7183f-a7bf-430b-b985-11df611c9c99 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:10.298381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7183f-a7bf-430b-b985-11df611c9c99 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:10.299733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 578/1167] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:10.304905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2d110-0f5c-4837-af72-917064c86d96 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:10.305314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2d110-0f5c-4837-af72-917064c86d96 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:10.305455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2d110-0f5c-4837-af72-917064c86d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:10.323740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3ca2d110-0f5c-4837-af72-917064c86d96 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:10.324913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3ca2d110-0f5c-4837-af72-917064c86d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:10.390665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 590/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:10.413118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7183f-a7bf-430b-b985-11df611c9c99 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:10.469398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:10.513233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7183f-a7bf-430b-b985-11df611c9c99 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:10.545814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c3527-1878-413a-a66e-e96921e6dadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:10.546449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3527-1878-413a-a66e-e96921e6dadb None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:10.546728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3527-1878-413a-a66e-e96921e6dadb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:10.547001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c3527-1878-413a-a66e-e96921e6dadb None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:10.547697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c3527-1878-413a-a66e-e96921e6dadb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:10.549153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c3527-1878-413a-a66e-e96921e6dadb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:10.620102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 579/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Wed Jul 27 08:22:10 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:22:12.618987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fc9fa-e826-4a4f-a4dd-b7ea92a6dca1 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:12.619482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fc9fa-e826-4a4f-a4dd-b7ea92a6dca1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:12.619594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fc9fa-e826-4a4f-a4dd-b7ea92a6dca1 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:12.620876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 591/1170] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:12.626573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb3689-87d4-416f-bc61-170b8f2c5c5d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:12.626993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb3689-87d4-416f-bc61-170b8f2c5c5d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:12.627127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb3689-87d4-416f-bc61-170b8f2c5c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:12.645221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2bdb3689-87d4-416f-bc61-170b8f2c5c5d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:12.646481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2bdb3689-87d4-416f-bc61-170b8f2c5c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:12.714593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 580/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:12.738668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fc9fa-e826-4a4f-a4dd-b7ea92a6dca1 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:12.754174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:12.836396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fc9fa-e826-4a4f-a4dd-b7ea92a6dca1 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:12.869554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf39d49-b627-4170-9c16-c40db467a5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:12.870190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf39d49-b627-4170-9c16-c40db467a5dc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:12.870476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf39d49-b627-4170-9c16-c40db467a5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:12.870751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf39d49-b627-4170-9c16-c40db467a5dc None admin] PATH_INFO: `/v3/domains/3002a0846c144358962121ab8d17ce09` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:12.881160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf39d49-b627-4170-9c16-c40db467a5dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=3002a0846c144358962121ab8d17ce09)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:12.882653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf39d49-b627-4170-9c16-c40db467a5dc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:12.884244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 592/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Jul 27 08:22:12 2022] GET /identity/v3/domains/3002a0846c144358962121ab8d17ce09 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:12.890405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb3689-87d4-416f-bc61-170b8f2c5c5d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:12.934963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:12.985525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb3689-87d4-416f-bc61-170b8f2c5c5d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:13.018265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907a4ee-d6be-4a9b-a6dc-f3502cc4ac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:13.018872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907a4ee-d6be-4a9b-a6dc-f3502cc4ac37 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:13.019148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907a4ee-d6be-4a9b-a6dc-f3502cc4ac37 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:13.019415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907a4ee-d6be-4a9b-a6dc-f3502cc4ac37 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:13.020092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907a4ee-d6be-4a9b-a6dc-f3502cc4ac37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:13.021509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907a4ee-d6be-4a9b-a6dc-f3502cc4ac37 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:13.098110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 581/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:22:12 2022] POST /identity/v3/projects => generated 361 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:22:15.090215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd5a6-40d5-4d85-9a24-ce587fdbf7d1 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:15.091266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd5a6-40d5-4d85-9a24-ce587fdbf7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:15.091366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd5a6-40d5-4d85-9a24-ce587fdbf7d1 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:15.092829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 593/1174] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:15.098164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1867a-50a1-4ef5-9df3-24615bae806b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:15.098555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1867a-50a1-4ef5-9df3-24615bae806b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:15.098726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1867a-50a1-4ef5-9df3-24615bae806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:15.116737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-54a1867a-50a1-4ef5-9df3-24615bae806b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:15.117917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54a1867a-50a1-4ef5-9df3-24615bae806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:15.182870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 582/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:15.206406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdd5a6-40d5-4d85-9a24-ce587fdbf7d1 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:15.280664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:15.304601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdd5a6-40d5-4d85-9a24-ce587fdbf7d1 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:15.336613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906be3d8-81b3-41d0-9e05-82944a20807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:15.337229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be3d8-81b3-41d0-9e05-82944a20807e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:15.337505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be3d8-81b3-41d0-9e05-82944a20807e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:15.337775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be3d8-81b3-41d0-9e05-82944a20807e None admin] PATH_INFO: `/v3/domains/3002a0846c144358962121ab8d17ce09` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:15.338917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906be3d8-81b3-41d0-9e05-82944a20807e None admin] RBAC: Authorizing `identity:get_domain(domain_id=3002a0846c144358962121ab8d17ce09)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:15.340381 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906be3d8-81b3-41d0-9e05-82944a20807e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:15.342549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 594/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Jul 27 08:22:15 2022] GET /identity/v3/domains/3002a0846c144358962121ab8d17ce09 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:15.348628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1867a-50a1-4ef5-9df3-24615bae806b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:15.376119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:15.446785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1867a-50a1-4ef5-9df3-24615bae806b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:15.478968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d4588-aa12-442d-8b48-eaa705d8bce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:15.479571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4588-aa12-442d-8b48-eaa705d8bce0 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:15.479846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4588-aa12-442d-8b48-eaa705d8bce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:15.480114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4588-aa12-442d-8b48-eaa705d8bce0 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:15.480779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d4588-aa12-442d-8b48-eaa705d8bce0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:15.482771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d4588-aa12-442d-8b48-eaa705d8bce0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:15.585026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 583/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:22:15 2022] POST /identity/v3/users => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:22:17.615727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919706-c5e3-4246-8ad7-a4bdf4b1ec1e None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:17.616215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919706-c5e3-4246-8ad7-a4bdf4b1ec1e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:17.616306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919706-c5e3-4246-8ad7-a4bdf4b1ec1e None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:17.617614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 595/1178] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:17.622815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19910910-707e-412f-9690-db8df0a65fce None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:17.623219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19910910-707e-412f-9690-db8df0a65fce None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:17.623356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19910910-707e-412f-9690-db8df0a65fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:17.641259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-19910910-707e-412f-9690-db8df0a65fce None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:17.642450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-19910910-707e-412f-9690-db8df0a65fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:17.705792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 584/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:17.728971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919706-c5e3-4246-8ad7-a4bdf4b1ec1e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:17.828701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919706-c5e3-4246-8ad7-a4bdf4b1ec1e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:17.862419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:17.863036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:17.863313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:17.863579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:17.874371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:22:17.879107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:22:17.880203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:22:17.881250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 596/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:22:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:22:17.886371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19910910-707e-412f-9690-db8df0a65fce None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:17.901478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:17.980955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19910910-707e-412f-9690-db8df0a65fce None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:18.015159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:18.015757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:18.016034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:18.016302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:18.017001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:18.017301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:18.018691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:18.026549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 585/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:22:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:18.032164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:18.035050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:18.130016 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6124ec-a749-4ac3-8d97-ad9206e8f7cf None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:18.162376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:18.162990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:18.163263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:18.163531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] PATH_INFO: `/v3/users/711921d5c1db46419f96c0dd31d54061` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:18.176666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] RBAC: Authorizing `identity:get_user(user_id=711921d5c1db46419f96c0dd31d54061)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:18.178084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:18.179695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 597/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:22:18 2022] GET /identity/v3/users/711921d5c1db46419f96c0dd31d54061 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:18.185504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:18.259532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:18.282638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161517ba-a935-4e1e-9742-7d302947b8a3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:18.314835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:18.315441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:18.315721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:18.315996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:18.380231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:18.381608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:18.454750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:18.477363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 586/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:22:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:22:18.483739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:18.540223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:18.580174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f6bee-d5b2-4058-9a5c-063ede4e3400 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:18.613956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede747ed-cf63-4ca3-8581-0b7669dbc150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:18.614610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede747ed-cf63-4ca3-8581-0b7669dbc150 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:18.614890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede747ed-cf63-4ca3-8581-0b7669dbc150 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:18.615258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede747ed-cf63-4ca3-8581-0b7669dbc150 None admin] PATH_INFO: `/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:18.616394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede747ed-cf63-4ca3-8581-0b7669dbc150 None admin] RBAC: Authorizing `identity:get_project(project_id=f79ff13b62bf49e38fb3ccb95721cd34)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:18.617841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede747ed-cf63-4ca3-8581-0b7669dbc150 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:18.619559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 598/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:22:18 2022] GET /identity/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:18.625667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:18.721713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029aad-f73b-4fdf-a3d1-715117ab5b49 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:18.755171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006e0e4-7cff-497f-8b47-bae23db43e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:18.755774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e0e4-7cff-497f-8b47-bae23db43e40 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:18.756052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e0e4-7cff-497f-8b47-bae23db43e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:18.756324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006e0e4-7cff-497f-8b47-bae23db43e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:18.757100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006e0e4-7cff-497f-8b47-bae23db43e40 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=f79ff13b62bf49e38fb3ccb95721cd34, user.id=711921d5c1db46419f96c0dd31d54061) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:18.757411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006e0e4-7cff-497f-8b47-bae23db43e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:18.758856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006e0e4-7cff-497f-8b47-bae23db43e40 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:18.764161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 587/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:22:18 2022] GET /identity/v3/role_assignments?user.id=711921d5c1db46419f96c0dd31d54061&scope.project.id=f79ff13b62bf49e38fb3ccb95721cd34&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:20.725255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c51bdc-2875-4d3c-9daf-6cdf34eaf638 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:20.725795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c51bdc-2875-4d3c-9daf-6cdf34eaf638 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:20.725795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c51bdc-2875-4d3c-9daf-6cdf34eaf638 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:20.727165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 599/1186] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:20.733013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8929-46aa-4112-9261-2ca854765cea None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:20.733399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8929-46aa-4112-9261-2ca854765cea None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:20.733553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad8929-46aa-4112-9261-2ca854765cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:20.751924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a3ad8929-46aa-4112-9261-2ca854765cea None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:20.753103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a3ad8929-46aa-4112-9261-2ca854765cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:20.785669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:20.817879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 588/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:20.839766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c51bdc-2875-4d3c-9daf-6cdf34eaf638 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:20.938766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c51bdc-2875-4d3c-9daf-6cdf34eaf638 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:20.963462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:20.971605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:20.973002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:20.973002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:20.973002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:20.983321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:22:20.988512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:22:20.989509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:22:20.990742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 600/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:22:21.000257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8929-46aa-4112-9261-2ca854765cea None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:21.094282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad8929-46aa-4112-9261-2ca854765cea None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:21.126721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:21.127315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:21.127590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:21.127857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:21.128553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:21.128848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:21.130218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:21.138090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 589/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:21.143747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:21.239512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0e4a4-ac18-4dd6-ab28-23a6bb48b79a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:21.272040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:21.272651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:21.272922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:21.273188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] PATH_INFO: `/v3/users/711921d5c1db46419f96c0dd31d54061` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:21.274342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] RBAC: Authorizing `identity:get_user(user_id=711921d5c1db46419f96c0dd31d54061)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:21.275751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:21.277301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 601/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:22:21 2022] GET /identity/v3/users/711921d5c1db46419f96c0dd31d54061 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:21.283389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:21.313669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:21.380078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7943f-8fab-4440-90b0-ace979e22a9d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:21.412606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcb296-2e94-4cb4-bbbd-aee8976e7b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:21.413227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcb296-2e94-4cb4-bbbd-aee8976e7b8e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:21.413506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcb296-2e94-4cb4-bbbd-aee8976e7b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:21.413776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcb296-2e94-4cb4-bbbd-aee8976e7b8e None admin] PATH_INFO: `/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:21.414931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcb296-2e94-4cb4-bbbd-aee8976e7b8e None admin] RBAC: Authorizing `identity:get_project(project_id=f79ff13b62bf49e38fb3ccb95721cd34)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:21.416330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcb296-2e94-4cb4-bbbd-aee8976e7b8e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:21.417945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 590/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:22:21 2022] GET /identity/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:22:21.424251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:21.478813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:21.521711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c307e77-ef02-4846-b5e4-e387b16178ea None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:21.554621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc119ad0-3e2e-40e8-8f1d-b03443eb12e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:21.555276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc119ad0-3e2e-40e8-8f1d-b03443eb12e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:21.555554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc119ad0-3e2e-40e8-8f1d-b03443eb12e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:21.555826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc119ad0-3e2e-40e8-8f1d-b03443eb12e4 None admin] PATH_INFO: `/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34/users/711921d5c1db46419f96c0dd31d54061/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:21.557731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc119ad0-3e2e-40e8-8f1d-b03443eb12e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f79ff13b62bf49e38fb3ccb95721cd34, user_id=711921d5c1db46419f96c0dd31d54061, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:21.559838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc119ad0-3e2e-40e8-8f1d-b03443eb12e4 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:21.613085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 602/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:22:21 2022] PUT /identity/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34/users/711921d5c1db46419f96c0dd31d54061/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:22:23.615024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dccd42-e447-449c-a5e3-b948125c1b32 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:23.615548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dccd42-e447-449c-a5e3-b948125c1b32 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:23.615548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dccd42-e447-449c-a5e3-b948125c1b32 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:23.616898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 591/1193] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:22:23.622828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59c74-3e49-499d-904e-721b78129ecf None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:23.623260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59c74-3e49-499d-904e-721b78129ecf None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:23.623362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59c74-3e49-499d-904e-721b78129ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:23.641080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c1a59c74-3e49-499d-904e-721b78129ecf None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:22:23.642288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c1a59c74-3e49-499d-904e-721b78129ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:22:23.707975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 603/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:22:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:22:23.731775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dccd42-e447-449c-a5e3-b948125c1b32 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:23.827842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:23.832285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dccd42-e447-449c-a5e3-b948125c1b32 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:23.865410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:23.866035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:23.866339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:23.866613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:23.877043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:22:23.881880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:22:23.883065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:22:23.884163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 592/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:22:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:22:23.889750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59c74-3e49-499d-904e-721b78129ecf None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:23.935839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:23.986645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59c74-3e49-499d-904e-721b78129ecf None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:24.018783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:24.019379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:24.019655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:24.019925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:24.020653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:24.020958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:24.022333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:24.030849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 604/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:22:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:24.036945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:24.136117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e2cd6-e48d-4e55-83b2-33b33a3d08c0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:24.169995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:24.170643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:24.170918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:24.171188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] PATH_INFO: `/v3/users/711921d5c1db46419f96c0dd31d54061` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:24.172327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] RBAC: Authorizing `identity:get_user(user_id=711921d5c1db46419f96c0dd31d54061)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:24.173753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:24.175367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 593/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 27 08:22:24 2022] GET /identity/v3/users/711921d5c1db46419f96c0dd31d54061 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:24.181100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:24.277497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1aab-d05a-4c9e-a2a3-d51c867ba9ea None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:24.285950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:24.310020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:24.310649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:24.310920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:24.311185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:24.376376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:24.377811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:24.472869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 605/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:22:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:22:24.478727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:24.507293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:24.578086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9222798-1503-4ba7-a0c1-f68459b7f999 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:24.611216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3719dd-7d3b-498c-be13-d06d211a4fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:24.611838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3719dd-7d3b-498c-be13-d06d211a4fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:24.612115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3719dd-7d3b-498c-be13-d06d211a4fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:24.612386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3719dd-7d3b-498c-be13-d06d211a4fa4 None admin] PATH_INFO: `/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:24.613514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3719dd-7d3b-498c-be13-d06d211a4fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=f79ff13b62bf49e38fb3ccb95721cd34)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:24.614959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3719dd-7d3b-498c-be13-d06d211a4fa4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:24.616591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 594/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 27 08:22:24 2022] GET /identity/v3/projects/f79ff13b62bf49e38fb3ccb95721cd34 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:24.622697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:22:24.632428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:24.717005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc1f11-8c5f-4cc3-940d-c778487edec8 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:22:24.749376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5d6f0-9c12-4653-b83f-1dd96733b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:22:24.749910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5d6f0-9c12-4653-b83f-1dd96733b02c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:22:24.750211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5d6f0-9c12-4653-b83f-1dd96733b02c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:22:24.750483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5d6f0-9c12-4653-b83f-1dd96733b02c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:22:24.751255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5d6f0-9c12-4653-b83f-1dd96733b02c None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=f79ff13b62bf49e38fb3ccb95721cd34, user.id=711921d5c1db46419f96c0dd31d54061) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:22:24.751559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5d6f0-9c12-4653-b83f-1dd96733b02c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:22:24.752971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5d6f0-9c12-4653-b83f-1dd96733b02c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:22:24.758522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 606/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:22:24 2022] GET /identity/v3/role_assignments?user.id=711921d5c1db46419f96c0dd31d54061&scope.project.id=f79ff13b62bf49e38fb3ccb95721cd34&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:22:34.607133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=52 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:22:34.755997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=64 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:36.923978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e653b2-5e30-4b2c-9a39-f38b2600c967 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:36.924588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e653b2-5e30-4b2c-9a39-f38b2600c967 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:36.924588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e653b2-5e30-4b2c-9a39-f38b2600c967 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:36.925883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 595/1201] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:23:36.931055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803effe-b28d-4ac8-b64a-54c9fe48cedb None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:36.933973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803effe-b28d-4ac8-b64a-54c9fe48cedb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:36.933973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803effe-b28d-4ac8-b64a-54c9fe48cedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:36.949581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4803effe-b28d-4ac8-b64a-54c9fe48cedb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:23:36.950799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4803effe-b28d-4ac8-b64a-54c9fe48cedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:23:37.018307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 607/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:23:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:23:37.044019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e653b2-5e30-4b2c-9a39-f38b2600c967 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:37.145890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e653b2-5e30-4b2c-9a39-f38b2600c967 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:37.178951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:37.179542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:37.179854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:37.180129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:37.191158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:37.192647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:37.207558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:37.208758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:37.209967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:37.209967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:37.209967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:37.209967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:37.209967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 596/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:37.214995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803effe-b28d-4ac8-b64a-54c9fe48cedb None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:37.312697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803effe-b28d-4ac8-b64a-54c9fe48cedb None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:37.340491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:37.346441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b6995-e1ee-40f4-87f1-3a3a13ba1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:37.349014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6995-e1ee-40f4-87f1-3a3a13ba1c17 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:37.349014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6995-e1ee-40f4-87f1-3a3a13ba1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:37.349014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b6995-e1ee-40f4-87f1-3a3a13ba1c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:37.349014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6995-e1ee-40f4-87f1-3a3a13ba1c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:37.349014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6995-e1ee-40f4-87f1-3a3a13ba1c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:37.350365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b6995-e1ee-40f4-87f1-3a3a13ba1c17 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:37.361775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 608/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:37.367792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:37.422113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:37.469060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52844a6c-1c78-4c2e-9061-2dab3908c193 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:37.502541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acaeaf-180e-4621-b57b-9f0122f588a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:37.503236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acaeaf-180e-4621-b57b-9f0122f588a0 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:37.503520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acaeaf-180e-4621-b57b-9f0122f588a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:37.503793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acaeaf-180e-4621-b57b-9f0122f588a0 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:37.504459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acaeaf-180e-4621-b57b-9f0122f588a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:37.506186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acaeaf-180e-4621-b57b-9f0122f588a0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:37.608580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 597/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 27 08:23:37 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:23:39.614533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e680-3e0d-4c0e-87c3-6f11332a0785 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:39.615064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e680-3e0d-4c0e-87c3-6f11332a0785 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:39.615064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3e680-3e0d-4c0e-87c3-6f11332a0785 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:39.616414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 609/1206] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:23:39.621442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc8cf-26fe-4c23-aba4-46e578820f0f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:39.621872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc8cf-26fe-4c23-aba4-46e578820f0f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:39.622044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fc8cf-26fe-4c23-aba4-46e578820f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:39.640516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-dc6fc8cf-26fe-4c23-aba4-46e578820f0f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:23:39.641716 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-dc6fc8cf-26fe-4c23-aba4-46e578820f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:23:39.706592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 598/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:23:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:23:39.729514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e680-3e0d-4c0e-87c3-6f11332a0785 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:39.829102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3e680-3e0d-4c0e-87c3-6f11332a0785 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:39.862834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:39.863467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:39.863748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:39.864024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:39.875405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:23:39.880302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:39.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:39.885877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 610/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:39.887438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fc8cf-26fe-4c23-aba4-46e578820f0f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:39.926619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:39.984696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fc8cf-26fe-4c23-aba4-46e578820f0f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:40.016614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:40.017225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:40.017502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:40.017777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:40.018508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:40.018818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:40.020177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:40.027774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 599/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:23:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:40.034250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:40.085305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:40.134896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e998-2a33-41df-9436-15a95c54377b None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:40.168851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:40.169824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:40.170307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:40.170749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:40.181494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:40.182945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:40.197356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:40.198521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:40.199564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:40.199564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:40.199564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:40.199564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:40.199564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 611/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:40.205130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:40.300782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:40.303644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9feb0-f1ed-44f3-95fe-e27f6d83dfac None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:40.336853 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:40.337514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:40.337904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:40.338309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:40.339167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:40.339567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:40.341058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:40.353217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 600/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:40.359379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:40.455741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8de0e-38eb-4f06-9361-f5151ebf3892 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:40.457231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:40.488621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:40.489242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:40.489517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:40.489786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:40.496971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:40.499036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:23:40.509823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:40.511108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:40.512212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 612/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:40.516807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:40.615818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf055b8-f915-4739-b21e-39ac4fe7d5cf None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:40.651345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:40.651966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:40.652212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:40.652479 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:40.653256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:40.653554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:40.655314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:40.672404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 601/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 27 08:23:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:40.678266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:40.777940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579c195-2710-416e-a967-01a6ebe5a168 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:40.810116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:40.811301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:40.811734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:40.812006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:40.841082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:40.879917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:40.881276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:40.977863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 613/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:23:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:23:40.983948 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:41.002359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:41.081908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b876d-cf97-442c-a5e6-37d15c1a1c9a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:41.116145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:41.117946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:41.117946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:41.117946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:41.127224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:41.128676 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:41.143601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:41.144673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:41.145833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:41.145833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:41.145833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:41.145833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:41.145833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 602/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:41.150465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:41.212166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:41.250769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab049b12-e843-4e0e-9bcb-e828a19715eb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:41.284172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:41.284768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:41.285045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:41.285317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:41.286003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:41.286335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:41.287692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:41.298508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 614/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:41.304197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:41.371221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:41.401594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf6dfc-3ff5-4deb-a060-da78b82a78a3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:41.433788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:41.434484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:41.434768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:41.435045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:41.442280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:41.443836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:23:41.456187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:41.457273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:41.458663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 603/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:23:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:23:41.463115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:41.562498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c675b-6bca-472f-8ace-a15bc1851a89 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:41.585336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:41.597059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecfc1b-28ad-4360-ba67-a15e079bd07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:41.597671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecfc1b-28ad-4360-ba67-a15e079bd07e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:41.597952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecfc1b-28ad-4360-ba67-a15e079bd07e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:41.598255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecfc1b-28ad-4360-ba67-a15e079bd07e None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:41.599006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecfc1b-28ad-4360-ba67-a15e079bd07e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:41.599316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecfc1b-28ad-4360-ba67-a15e079bd07e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:41.600753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecfc1b-28ad-4360-ba67-a15e079bd07e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:41.612553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 615/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:23:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:41.618825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:41.719765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f45d8b-6035-434d-b316-05207cb5f689 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:41.747390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:41.753682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78238aa-d71a-4cea-ab93-42ff5f5a84ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:41.754326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78238aa-d71a-4cea-ab93-42ff5f5a84ed None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:41.754606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78238aa-d71a-4cea-ab93-42ff5f5a84ed None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:41.754876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78238aa-d71a-4cea-ab93-42ff5f5a84ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:41.755663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78238aa-d71a-4cea-ab93-42ff5f5a84ed None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=04d05e0dfd474dfdb848b129941532b5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:41.755971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78238aa-d71a-4cea-ab93-42ff5f5a84ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:41.757384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78238aa-d71a-4cea-ab93-42ff5f5a84ed None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:41.762664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 604/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:23:41 2022] GET /identity/v3/role_assignments?user.id=04d05e0dfd474dfdb848b129941532b5&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:43.728931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfea0d-12db-4ddf-b4f2-d82e428cfafd None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:43.729450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfea0d-12db-4ddf-b4f2-d82e428cfafd None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:43.729560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfea0d-12db-4ddf-b4f2-d82e428cfafd None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:43.730942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 616/1220] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:23:43.736627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790e789-efc8-4616-95e6-3d52aa7ec52d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:43.737070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790e789-efc8-4616-95e6-3d52aa7ec52d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:43.737176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790e789-efc8-4616-95e6-3d52aa7ec52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:43.754997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6790e789-efc8-4616-95e6-3d52aa7ec52d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:23:43.759252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6790e789-efc8-4616-95e6-3d52aa7ec52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:23:43.821182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 605/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:23:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:23:43.844104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfea0d-12db-4ddf-b4f2-d82e428cfafd None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:43.943045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfea0d-12db-4ddf-b4f2-d82e428cfafd None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:43.976904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:43.977530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:43.977806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:43.978079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:43.988009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:23:43.992902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:43.994053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:43.994936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 617/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:23:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:44.000524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790e789-efc8-4616-95e6-3d52aa7ec52d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:44.098645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790e789-efc8-4616-95e6-3d52aa7ec52d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:44.121500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:44.132324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:44.132933 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:44.133209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:44.133476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:44.134209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:44.134511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:44.135866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:44.144472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 606/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:23:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:44.150261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:44.194514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:44.249424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554df88f-c420-40c8-ad19-d20264e40d80 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:44.281316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:44.282565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:44.289293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:44.289293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:44.292700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:44.294191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:44.309173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:44.310262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:44.311446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:44.311446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:44.311446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:44.311446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:44.311446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 618/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:44.318484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:44.415935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f80708-9c61-46d7-bbbf-ac78aed25967 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:44.449520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:44.450121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:44.450430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:44.450701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:44.451378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:44.451680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:44.453020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:44.464983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 607/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:44.471227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:44.561773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:44.569197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd18133-530c-4138-9c7c-52c2a2b99290 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:44.601892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:44.602539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:44.602889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:44.603134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:44.610863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:44.612303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:23:44.623237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:44.624397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:44.626316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 619/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:44.630966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:44.678528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:44.731016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab128060-f10c-4bcb-8cc0-bdf0dce49290 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:44.766128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:44.766786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:44.767022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:44.767290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:44.768123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:44.768429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:44.769833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:44.786263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 608/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 27 08:23:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:44.791948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:44.889375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f66c6-8122-451f-becd-8d2e2d25c174 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:44.923577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:44.924211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:44.924493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:44.924764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:44.935225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:44.936613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:44.950506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:44.951481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:44.952971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:44.952971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:44.952971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:44.952971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:44.952971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 620/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:44.957910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:45.055243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38158ab2-f597-4fbc-9027-e52d0cef00db None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:45.057517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:45.088997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:45.089603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:45.089880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:45.090201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:45.090886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:45.091186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:45.092787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:45.103600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 609/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:45.109358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:45.118530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:45.207499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3760a6-8fb1-4619-b2b5-d54c74d21acc None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:45.240276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:45.240905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:45.241203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:45.241456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:45.248044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:45.249476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:23:45.261092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:45.262324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:45.263460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 621/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 27 08:23:45.268866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:45.369481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783c530-6bd7-42ce-bae1-705df805fc2f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:45.404122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da345621-2559-40e9-a1e5-991f716af059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:45.404840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da345621-2559-40e9-a1e5-991f716af059 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:45.405009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da345621-2559-40e9-a1e5-991f716af059 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:45.405278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da345621-2559-40e9-a1e5-991f716af059 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:45.406018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da345621-2559-40e9-a1e5-991f716af059 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:45.406426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da345621-2559-40e9-a1e5-991f716af059 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:45.408419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da345621-2559-40e9-a1e5-991f716af059 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:45.420587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 610/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:45.427171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:45.492199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:45.526044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13db71-1592-498f-a4f8-d741e8f98b7a None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:45.559557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828049bc-8ccb-4274-8931-9aea6ae935bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:45.560209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828049bc-8ccb-4274-8931-9aea6ae935bc None admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:45.560484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828049bc-8ccb-4274-8931-9aea6ae935bc None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:45.560754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828049bc-8ccb-4274-8931-9aea6ae935bc None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/04d05e0dfd474dfdb848b129941532b5/roles/b2f748f19ea54f688475e569b0847642` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:45.575637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828049bc-8ccb-4274-8931-9aea6ae935bc None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=04d05e0dfd474dfdb848b129941532b5, role_id=b2f748f19ea54f688475e569b0847642)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:45.577077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828049bc-8ccb-4274-8931-9aea6ae935bc None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:45.669036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 622/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:23:45 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/04d05e0dfd474dfdb848b129941532b5/roles/b2f748f19ea54f688475e569b0847642 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:23:47.652164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c491ae-0c48-4daa-9bcd-88ea0fc08d85 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:47.652666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c491ae-0c48-4daa-9bcd-88ea0fc08d85 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:47.652781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c491ae-0c48-4daa-9bcd-88ea0fc08d85 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:47.654150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 611/1233] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:23:47.659465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2587c5b-051c-461f-a8f9-2a1435dc7a41 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:47.659867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2587c5b-051c-461f-a8f9-2a1435dc7a41 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:47.660026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2587c5b-051c-461f-a8f9-2a1435dc7a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:47.678045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c2587c5b-051c-461f-a8f9-2a1435dc7a41 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:23:47.679242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c2587c5b-051c-461f-a8f9-2a1435dc7a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:23:47.743532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 623/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:23:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:23:47.766704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c491ae-0c48-4daa-9bcd-88ea0fc08d85 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:47.791302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:47.864606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c491ae-0c48-4daa-9bcd-88ea0fc08d85 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:47.897459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:47.898109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:47.898421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:47.898694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:47.910514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:23:47.915458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:47.916671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:47.917700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 612/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 27 08:23:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:47.923320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2587c5b-051c-461f-a8f9-2a1435dc7a41 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:47.968792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:48.020448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2587c5b-051c-461f-a8f9-2a1435dc7a41 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:48.054251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:48.054865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:48.055146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:48.055420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:48.056171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:48.056519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:48.057922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:48.065697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 624/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 27 08:23:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:23:48.071464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:48.153993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:48.169713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c33b8-95ab-4355-a964-2921357abf7e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:48.203618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:48.204393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:48.204674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:48.204947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:48.216095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:48.217876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:48.232443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:48.233472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:48.250382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:48.250382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:48.250382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:48.250382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:48.250382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 613/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:48.250741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:48.337721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc244d-2eef-4991-802e-09fbf47d7f88 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:48.347648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:48.371944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:48.372554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:48.372839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:48.373112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:48.373839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:48.374166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:48.375554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:48.386772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 625/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:48.392942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:48.495441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5401bc-b28d-4ddb-bc0c-c00fd0c29fb1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:48.528765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:48.529392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:48.529671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:48.529941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:48.532657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:48.538298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:48.539808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:23:48.551531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:48.552699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:48.553831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 614/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:48.558782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:48.659291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc2f5b-fb74-4aa9-b890-c3040d243874 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:48.691818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:48.692419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:48.692690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:48.692964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:48.693752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:48.694058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:48.695518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:48.713146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 626/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 27 08:23:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:48.722128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:48.823222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beac41c-9bf3-4e60-a78d-0801210bdf95 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:48.856572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:48.857181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:48.857459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:48.857726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:48.916997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:48.923328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:48.924782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:49.022430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 615/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:23:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:23:49.028240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:49.055091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:49.127234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e67e93-0148-4723-8305-78c91a7b0825 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:49.161486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:49.162117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:49.162431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:49.162699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:49.173173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:49.174613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:49.189309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:49.190487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:49.195441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:49.195441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:49.195441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:49.195441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:49.195441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 627/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:49.197058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:49.288647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:49.294350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cbb6-16f0-4d19-b6fa-77b0434baf30 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:49.328087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:49.328690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:49.328966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:49.329233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:49.329905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:49.330230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:49.331687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:49.342717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 616/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:49.348665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:49.431066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:49.456441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee21644-3d27-4bf5-a4ca-4c6358475415 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:49.491035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:49.491753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:49.492035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:49.492346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:49.499704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:49.501728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:23:49.513208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:49.514395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:49.518955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 628/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:49.520815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:49.630766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720bc56c-e0e5-4ef2-9e3d-30da61954b27 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:49.665862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4284bca-a701-49ec-ac76-4ab50aea0317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:49.666502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4284bca-a701-49ec-ac76-4ab50aea0317 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:49.666786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4284bca-a701-49ec-ac76-4ab50aea0317 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:49.667061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4284bca-a701-49ec-ac76-4ab50aea0317 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:49.667810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4284bca-a701-49ec-ac76-4ab50aea0317 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:49.668116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4284bca-a701-49ec-ac76-4ab50aea0317 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:49.669537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4284bca-a701-49ec-ac76-4ab50aea0317 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:49.683066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 617/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:49.689509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:49.807696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42929-fdbf-41fd-a691-87cd09f341fb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:49.844041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7167a-3ccd-4fa8-8268-e556db9f1134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:49.844743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7167a-3ccd-4fa8-8268-e556db9f1134 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:49.845023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7167a-3ccd-4fa8-8268-e556db9f1134 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:49.845297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7167a-3ccd-4fa8-8268-e556db9f1134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:49.846082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7167a-3ccd-4fa8-8268-e556db9f1134 None admin] RBAC: Adding query filter params (role.id=b2f748f19ea54f688475e569b0847642, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=04d05e0dfd474dfdb848b129941532b5) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:49.846420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7167a-3ccd-4fa8-8268-e556db9f1134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:49.847856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7167a-3ccd-4fa8-8268-e556db9f1134 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:49.853702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 629/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:23:49 2022] GET /identity/v3/role_assignments?user.id=04d05e0dfd474dfdb848b129941532b5&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=b2f748f19ea54f688475e569b0847642 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:51.972389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3239d5-425d-4b51-b923-138542af8731 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:51.972926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3239d5-425d-4b51-b923-138542af8731 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:51.972926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3239d5-425d-4b51-b923-138542af8731 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:51.974291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 618/1247] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:23:51.979442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecacba-bbf5-4e20-acec-bf0e3958e4df None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:51.979824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecacba-bbf5-4e20-acec-bf0e3958e4df None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:51.979988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecacba-bbf5-4e20-acec-bf0e3958e4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:51.987033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:51.998867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-acecacba-bbf5-4e20-acec-bf0e3958e4df None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:23:52.000202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-acecacba-bbf5-4e20-acec-bf0e3958e4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:23:52.064947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 630/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:23:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:23:52.088635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3239d5-425d-4b51-b923-138542af8731 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:52.107424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:52.189430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3239d5-425d-4b51-b923-138542af8731 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:52.224338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:52.224925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:52.225199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:52.225465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:52.235331 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:23:52.240281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:52.241363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:52.242350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 619/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:52.247722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecacba-bbf5-4e20-acec-bf0e3958e4df None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:52.345664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecacba-bbf5-4e20-acec-bf0e3958e4df None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:52.374447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:52.379726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:52.380361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:52.380683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:52.380954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:52.381701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:52.382007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:52.383423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:52.392787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 631/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:52.398930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:52.476266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:52.497953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6dd1f-250b-4992-b052-a47de937b46b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:52.530474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:52.531869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:52.532149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:52.532420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:52.542654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:52.544084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:52.557841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:52.559044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:52.559987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:52.559987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:52.559987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:52.559987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:52.559987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 620/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:52.565589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:52.672405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a1ea4-f89f-4f38-aec1-0410b42b757d None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:52.705345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:52.705957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:52.706264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:52.706540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:52.707223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:52.707529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:52.708895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:52.721684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 632/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:52.731974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:52.824812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ea43b-fd1a-4c4e-88cf-d2d907eee0a7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:52.851280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:52.858944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:52.859739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:52.860062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:52.860346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:52.867707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:52.869135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:23:52.880723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:52.881815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:52.883099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 621/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:52.887792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:52.944058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:52.986490 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e57e-64c3-4852-94c6-c2e54e5cd2e1 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:53.019291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:53.019898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:53.020179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:53.020452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:53.021241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:53.021563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:53.023038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:53.039797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 633/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 27 08:23:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:53.045546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:53.145545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27991d7e-b07b-47a5-8de3-6645569b7a96 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:53.179374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:53.181242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:53.181242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:53.181242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:53.235718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:53.247790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:53.249197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:53.346915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 622/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:23:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:23:53.352752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:53.450165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e92ce0-dc8f-413b-9e7b-d7d34dbab509 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:53.463383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:53.485341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:53.485965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:53.486278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:53.486546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:53.496375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:53.497856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:53.512127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:53.513189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:53.514510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:53.514510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:53.514510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:53.514510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:53.514510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 634/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:53.519210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:53.606282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:53.618243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99d0aa-5384-4154-87d3-9e628ff2a519 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:53.651305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db58380-4bee-4913-811b-67904584d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:53.651910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db58380-4bee-4913-811b-67904584d53a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:53.652184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db58380-4bee-4913-811b-67904584d53a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:53.652450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db58380-4bee-4913-811b-67904584d53a None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:53.653174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db58380-4bee-4913-811b-67904584d53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:53.653469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db58380-4bee-4913-811b-67904584d53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:53.654848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db58380-4bee-4913-811b-67904584d53a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:53.666335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 623/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:53.672153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:53.788374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b5b70-5f86-4bf7-a635-a5c0cd1d3ccb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:53.821736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:53.822384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:53.822666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:53.822940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:53.829576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:53.832532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:23:53.846983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:53.864077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:23:53.865313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 635/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:23:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:53.866035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db58380-4bee-4913-811b-67904584d53a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:53.954656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db58380-4bee-4913-811b-67904584d53a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:53.987390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f7830-e613-44c3-8da8-36a88a91977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:53.987993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f7830-e613-44c3-8da8-36a88a91977e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:53.988294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f7830-e613-44c3-8da8-36a88a91977e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:53.988568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f7830-e613-44c3-8da8-36a88a91977e None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:53.989304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f7830-e613-44c3-8da8-36a88a91977e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:53.989607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f7830-e613-44c3-8da8-36a88a91977e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:53.991043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f7830-e613-44c3-8da8-36a88a91977e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:54.003313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 624/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:23:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:23:54.009532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:54.036358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:54.110435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eee778-d13c-43af-8576-c40974f71cc2 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:54.142735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887570-11ba-408c-a2ca-a9756579b2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:54.143336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887570-11ba-408c-a2ca-a9756579b2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:54.143613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887570-11ba-408c-a2ca-a9756579b2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:54.143884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887570-11ba-408c-a2ca-a9756579b2a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:54.144755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887570-11ba-408c-a2ca-a9756579b2a2 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=04d05e0dfd474dfdb848b129941532b5) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:54.145026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887570-11ba-408c-a2ca-a9756579b2a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:54.146490 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887570-11ba-408c-a2ca-a9756579b2a2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:54.151772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 636/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:23:54 2022] GET /identity/v3/role_assignments?user.id=04d05e0dfd474dfdb848b129941532b5&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:56.124107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa628e-4f37-42b4-ba52-3d2bc2b1a0a1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:56.125417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa628e-4f37-42b4-ba52-3d2bc2b1a0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:56.125417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa628e-4f37-42b4-ba52-3d2bc2b1a0a1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:56.126566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 625/1261] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:23:56.131795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b315e-e2ed-4662-9002-727c5a92d4c3 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:56.133927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b315e-e2ed-4662-9002-727c5a92d4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:56.133927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b315e-e2ed-4662-9002-727c5a92d4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:56.150326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ef5b315e-e2ed-4662-9002-727c5a92d4c3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:23:56.151451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ef5b315e-e2ed-4662-9002-727c5a92d4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:23:56.217620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 637/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:23:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:23:56.240384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa628e-4f37-42b4-ba52-3d2bc2b1a0a1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:56.252172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:56.346699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa628e-4f37-42b4-ba52-3d2bc2b1a0a1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:56.381899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:56.382519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:56.382801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:56.384418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:56.394337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:23:56.399141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:56.400284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:56.401529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 626/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:23:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:56.406865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b315e-e2ed-4662-9002-727c5a92d4c3 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:56.425429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:56.502670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b315e-e2ed-4662-9002-727c5a92d4c3 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:56.537181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:56.537783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:56.538060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:56.538356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:56.539110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:56.539404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:56.540760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:56.549283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 638/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:23:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:56.555044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:56.623914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:56.658630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f1c9f0-4427-438c-bd16-f290f728d2b0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:56.691491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:56.692113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:56.692388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:56.692658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:56.703625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:56.705049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:56.719772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:56.725265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:56.726177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:56.726177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:56.726177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:56.726177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:56.726177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 627/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:56.726769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:56.808378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:56.826964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e26dfc-ca74-4ad1-bbe4-376986b55e45 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:56.859876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:56.860474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:56.860751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:56.861020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:56.861807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:56.862106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:56.863486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:56.874778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 639/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:56.880558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:56.982272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c862e5-a6e3-4da1-8d36-81e993856076 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:57.002919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:57.017342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:57.017970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:57.018281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:57.018557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:57.026924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:57.028404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:23:57.040023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:57.041224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:57.042216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 628/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:57.047682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:57.145693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c8dd1-6077-45af-b61c-e24d15e33e3e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:57.179121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:57.179724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:57.180003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:57.180272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:57.181056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:57.181361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:57.182819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:57.191434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:57.200684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 640/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 27 08:23:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:57.206442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:57.308725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93464db3-3265-4d5c-a12d-88ddbc8fa907 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:57.343550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:57.344261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:57.344545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:57.344818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:57.355427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:57.356865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:23:57.371022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:57.372066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:57.373169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:23:57.373169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:23:57.373169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:23:57.373169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:57.373169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 629/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:57.378363 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:57.484774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b1ac6-8eef-4070-8d30-9cdfe615b897 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:57.520924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:57.521543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:57.521824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:57.522124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:57.522857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:57.523155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:57.524525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:57.536008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 641/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:57.541679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:57.575022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:57.643332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f367cd-ea1e-47a5-8242-c2cd59af213e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:57.678717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:57.679350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:57.679628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:57.679907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:57.686401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:57.688610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:23:57.699668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:23:57.708268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:23:57.709397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 630/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:23:57.709960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:57.766167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:57.807089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e8c61-d0b9-4f5a-9244-fb159276bacb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:57.838528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b818728-d7e0-498b-ae30-f9e6ee81d274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:57.839128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b818728-d7e0-498b-ae30-f9e6ee81d274 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:57.839458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b818728-d7e0-498b-ae30-f9e6ee81d274 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:57.839734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b818728-d7e0-498b-ae30-f9e6ee81d274 None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:57.840680 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b818728-d7e0-498b-ae30-f9e6ee81d274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:23:57.841183 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b818728-d7e0-498b-ae30-f9e6ee81d274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:57.844046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b818728-d7e0-498b-ae30-f9e6ee81d274 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:57.855551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 642/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:23:57.862483 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:23:57.952765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:23:57.961101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b521d8e6-9830-47aa-a75f-11cc075d3ecb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:23:57.995310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa636f-212a-4d2f-a501-986ad79ffea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:23:57.995968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa636f-212a-4d2f-a501-986ad79ffea8 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:23:57.996248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa636f-212a-4d2f-a501-986ad79ffea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:23:57.996521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa636f-212a-4d2f-a501-986ad79ffea8 None admin] PATH_INFO: `/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/04d05e0dfd474dfdb848b129941532b5/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:23:57.998462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa636f-212a-4d2f-a501-986ad79ffea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d120f709a1446e9614a10cfc7b01e2, user_id=04d05e0dfd474dfdb848b129941532b5, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:23:58.000203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa636f-212a-4d2f-a501-986ad79ffea8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:23:58.098559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 631/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 27 08:23:57 2022] PUT /identity/v3/projects/e9d120f709a1446e9614a10cfc7b01e2/users/04d05e0dfd474dfdb848b129941532b5/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:24:00.079981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882b4c-edb9-4af4-95d5-82813a756769 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:00.080563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882b4c-edb9-4af4-95d5-82813a756769 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:00.080563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882b4c-edb9-4af4-95d5-82813a756769 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:00.081914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 643/1274] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:24:00.086947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b0a6-b3db-40c2-84a9-0fdcfb553fa6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:00.087321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b0a6-b3db-40c2-84a9-0fdcfb553fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:00.087499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c03b0a6-b3db-40c2-84a9-0fdcfb553fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:00.106642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7c03b0a6-b3db-40c2-84a9-0fdcfb553fa6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:24:00.107841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7c03b0a6-b3db-40c2-84a9-0fdcfb553fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:24:00.174955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 632/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:24:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:24:00.198331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882b4c-edb9-4af4-95d5-82813a756769 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:00.296873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882b4c-edb9-4af4-95d5-82813a756769 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:00.315575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:00.331618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:00.332180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:00.332457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:00.332723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:00.343328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 27 08:24:00.348728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:00.353065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:00.354173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 644/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 27 08:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:00.356029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b0a6-b3db-40c2-84a9-0fdcfb553fa6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:00.438993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:00.457065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c03b0a6-b3db-40c2-84a9-0fdcfb553fa6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:00.490705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:00.491315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:00.491593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:00.491864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:00.492573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:00.492874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:00.494252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:00.503603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 633/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 27 08:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:00.509305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:00.610691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9205-2931-4bb5-8a3c-88664ec3c951 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:00.644989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:00.645639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:00.645931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:00.646225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:00.656051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:00.658333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:24:00.672895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:00.673949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:00.674933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:24:00.674933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:24:00.674933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:24:00.674933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:00.674933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 645/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:00.679835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:00.778455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895d9bd-bd72-458f-b568-ab78ac201233 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:00.811182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:00.811805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:00.812063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:00.812329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:00.813785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:00.814088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:00.815493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:00.826877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 634/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:00.833082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:00.865405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:00.932136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548ad8-c711-40f1-a3e9-46236168d7f9 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:00.965525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:00.966166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:00.970132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:00.970132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:00.974260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:00.975713 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 27 08:24:00.986535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:00.987488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:00.988520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 646/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:24:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:00.993732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:00.997620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:01.094972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ce72-a9ea-4b4a-b576-5fb3115c536b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:01.129755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:01.130333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:01.130610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:01.130883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:01.131672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:01.131981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:01.133399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:01.150273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 635/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 27 08:24:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:01.156142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:01.245465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:01.255359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b96b2b-c7f6-4144-929f-c692671f2c9e None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:01.287961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:01.288567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:01.288849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:01.289120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:01.356000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:01.357442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:01.445333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:01.455103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 647/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 27 08:24:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 27 08:24:01.460937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:01.524688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:01.562283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eced0be-5fc1-458c-b29c-583d3757d769 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:01.596438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:01.597064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:01.597340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:01.597609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:01.607353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:01.608772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 27 08:24:01.623584 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:01.627759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:01.628919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 27 08:24:01.628919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 27 08:24:01.628919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 27 08:24:01.628919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:24:01.628919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 636/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:01.630622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:01.728167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75e8a6-c8ba-45cf-b78f-3f2c58df1ff0 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:01.762918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:01.763525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:01.763806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:01.764079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:01.764758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:01.765058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:01.767315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:01.778436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 648/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 27 08:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:01.784385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:01.882621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56fd72-ebc9-4ec3-bdf8-dee1a5d14311 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:01.895656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:01.915006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:01.915627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:01.915906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:01.916178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:01.923281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:01.924691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 27 08:24:01.936892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:01.937974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 27 08:24:01.939824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Jul 27 08:24:01.942289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 637/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 27 08:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:01.947461 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:01.994795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:02.043802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778264-0746-4e85-ab18-e646a734865f None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:02.077563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f2b1d-e6ad-4fe2-97b1-b51662b1b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:02.078176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f2b1d-e6ad-4fe2-97b1-b51662b1b5aa None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:02.078457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f2b1d-e6ad-4fe2-97b1-b51662b1b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:02.078745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f2b1d-e6ad-4fe2-97b1-b51662b1b5aa None admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:02.079481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f2b1d-e6ad-4fe2-97b1-b51662b1b5aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:02.079781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f2b1d-e6ad-4fe2-97b1-b51662b1b5aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:02.081197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f2b1d-e6ad-4fe2-97b1-b51662b1b5aa None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:02.094036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 649/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 27 08:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:02.100547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:02.202416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad7d23-e83f-476b-b026-98dcf5eb0894 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:02.236629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a9572-8c30-4811-9484-d736be234ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:02.237236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9572-8c30-4811-9484-d736be234ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:02.237516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9572-8c30-4811-9484-d736be234ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:02.237786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a9572-8c30-4811-9484-d736be234ab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:02.238655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a9572-8c30-4811-9484-d736be234ab5 None admin] RBAC: Adding query filter params (role.id=3cf29058f0f54877aa0a7863c00bc628, scope.project.id=e9d120f709a1446e9614a10cfc7b01e2, user.id=04d05e0dfd474dfdb848b129941532b5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:02.238966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a9572-8c30-4811-9484-d736be234ab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:02.240455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a9572-8c30-4811-9484-d736be234ab5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:02.246352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 638/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 27 08:24:02 2022] GET /identity/v3/role_assignments?user.id=04d05e0dfd474dfdb848b129941532b5&scope.project.id=e9d120f709a1446e9614a10cfc7b01e2&role.id=3cf29058f0f54877aa0a7863c00bc628 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:04.275097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9eea-85c2-4c8d-9d91-0b8f4ca50db6 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:04.275633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9eea-85c2-4c8d-9d91-0b8f4ca50db6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:04.275726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da9eea-85c2-4c8d-9d91-0b8f4ca50db6 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:04.277037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 650/1288] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:24:04.282168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620c225-95bf-47da-b79c-e027268308cd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:04.282624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620c225-95bf-47da-b79c-e027268308cd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:04.282726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620c225-95bf-47da-b79c-e027268308cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:04.300835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5620c225-95bf-47da-b79c-e027268308cd None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:24:04.302005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5620c225-95bf-47da-b79c-e027268308cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:24:04.330742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:04.366703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 639/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:24:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:24:04.388389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da9eea-85c2-4c8d-9d91-0b8f4ca50db6 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:04.490485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da9eea-85c2-4c8d-9d91-0b8f4ca50db6 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:04.501485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:04.524445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:04.525070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:04.525403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:04.525629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:04.526276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:04.527664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:04.536354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:04.543678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:04.544759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 651/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:24:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:04.545737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620c225-95bf-47da-b79c-e027268308cd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:04.641588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620c225-95bf-47da-b79c-e027268308cd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:04.675086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14b606-231f-44a9-92be-952ac2d4f858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:04.675698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b606-231f-44a9-92be-952ac2d4f858 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:04.675977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b606-231f-44a9-92be-952ac2d4f858 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:04.676249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14b606-231f-44a9-92be-952ac2d4f858 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:04.676919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14b606-231f-44a9-92be-952ac2d4f858 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:04.677222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14b606-231f-44a9-92be-952ac2d4f858 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:04.678641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14b606-231f-44a9-92be-952ac2d4f858 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:04.683383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 640/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 27 08:24:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:04.689221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:04.787529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ea6c2-922e-4d99-8ed6-164c600905fb None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:04.819345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e68ac-9009-4134-960d-0ae95f64619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:04.819945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e68ac-9009-4134-960d-0ae95f64619e None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:04.820224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e68ac-9009-4134-960d-0ae95f64619e None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:04.820494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e68ac-9009-4134-960d-0ae95f64619e None admin] PATH_INFO: `/v3/services` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:04.821161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e68ac-9009-4134-960d-0ae95f64619e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:04.821470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e68ac-9009-4134-960d-0ae95f64619e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:04.822862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e68ac-9009-4134-960d-0ae95f64619e None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:04.827409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 652/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 27 08:24:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:06.857590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc3c1-b476-4602-9bf5-ecc0a6aed8c8 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:06.858158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc3c1-b476-4602-9bf5-ecc0a6aed8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:06.858158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc3c1-b476-4602-9bf5-ecc0a6aed8c8 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:06.860067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 641/1293] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:24:06.865220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c6f0-8227-4579-88e3-5415ada4f020 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:06.865625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c6f0-8227-4579-88e3-5415ada4f020 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:06.865779 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c6f0-8227-4579-88e3-5415ada4f020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:06.884478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c555c6f0-8227-4579-88e3-5415ada4f020 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:24:06.885774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c555c6f0-8227-4579-88e3-5415ada4f020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:24:06.915172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:06.952246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 653/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:24:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:24:06.974264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc3c1-b476-4602-9bf5-ecc0a6aed8c8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:07.005388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:07.074033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc3c1-b476-4602-9bf5-ecc0a6aed8c8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:07.106516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc83e-d504-4127-8d42-7d214d40c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:07.107120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc83e-d504-4127-8d42-7d214d40c700 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:07.107399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc83e-d504-4127-8d42-7d214d40c700 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:07.107718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc83e-d504-4127-8d42-7d214d40c700 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:07.108352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bc83e-d504-4127-8d42-7d214d40c700 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:07.110275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bc83e-d504-4127-8d42-7d214d40c700 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:07.175638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 642/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 27 08:24:06 2022] POST /identity/v3/services => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:24:09.199179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d67d68-58fa-4357-8f70-64a1b31f68b6 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:09.199716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d67d68-58fa-4357-8f70-64a1b31f68b6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:09.199716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d67d68-58fa-4357-8f70-64a1b31f68b6 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:09.201083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 654/1296] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:24:09.206272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae48cf-626f-46e0-a586-c6d1d7346b63 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:09.206673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae48cf-626f-46e0-a586-c6d1d7346b63 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:09.206814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae48cf-626f-46e0-a586-c6d1d7346b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:09.224870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bcae48cf-626f-46e0-a586-c6d1d7346b63 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:24:09.226045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bcae48cf-626f-46e0-a586-c6d1d7346b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:24:09.302955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 643/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:24:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:24:09.325737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d67d68-58fa-4357-8f70-64a1b31f68b6 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:09.428846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d67d68-58fa-4357-8f70-64a1b31f68b6 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:09.461917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:09.462516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:09.462791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:09.463059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:09.463676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:09.465036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:09.474281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:09.481645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:09.482805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 655/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:24:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:09.483773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae48cf-626f-46e0-a586-c6d1d7346b63 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:09.500276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:09.580450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae48cf-626f-46e0-a586-c6d1d7346b63 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:09.616003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:09.616613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:09.616892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:09.617160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:09.617833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:09.618126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:09.619492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:09.624607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 644/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 27 08:24:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:09.630694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:09.662293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:09.728605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49acaad-1a50-42b3-9151-aecfa75f2b21 None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:09.762037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232d9a8-8bf3-49e4-b51b-8dc5d5dc3cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:09.762711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232d9a8-8bf3-49e4-b51b-8dc5d5dc3cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:09.762989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232d9a8-8bf3-49e4-b51b-8dc5d5dc3cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:09.763261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232d9a8-8bf3-49e4-b51b-8dc5d5dc3cb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:09.764007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232d9a8-8bf3-49e4-b51b-8dc5d5dc3cb2 None admin] RBAC: Adding query filter params (interface=public, service_id=34f0e65f164d421bac4cdf04fe7ce7bd, region_id=RegionOne) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:09.764314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232d9a8-8bf3-49e4-b51b-8dc5d5dc3cb2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:09.765959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232d9a8-8bf3-49e4-b51b-8dc5d5dc3cb2 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:09.771457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 656/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 27 08:24:09 2022] GET /identity/v3/endpoints?service_id=34f0e65f164d421bac4cdf04fe7ce7bd&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:09.777154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:09.856957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:09.874551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055670bc-9575-42fb-b73e-2fd2a05c0897 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:09.908211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccba16c-1d0b-4838-b7cd-3322f547e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:09.908817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccba16c-1d0b-4838-b7cd-3322f547e62a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:09.909106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccba16c-1d0b-4838-b7cd-3322f547e62a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:09.909363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccba16c-1d0b-4838-b7cd-3322f547e62a None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:09.910002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccba16c-1d0b-4838-b7cd-3322f547e62a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:09.911388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccba16c-1d0b-4838-b7cd-3322f547e62a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:09.918036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 645/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 27 08:24:09 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:24:11.895750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474409e-0caf-4ab9-b315-04142f99839e None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:11.896281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474409e-0caf-4ab9-b315-04142f99839e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:11.896281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474409e-0caf-4ab9-b315-04142f99839e None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:11.897651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 657/1302] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:24:11.902781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce0eaf-c72f-43d9-97ef-e3887e0da4c9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:11.903220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce0eaf-c72f-43d9-97ef-e3887e0da4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:11.903315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce0eaf-c72f-43d9-97ef-e3887e0da4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:11.921342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d5ce0eaf-c72f-43d9-97ef-e3887e0da4c9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:24:11.923232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d5ce0eaf-c72f-43d9-97ef-e3887e0da4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:24:11.988765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 646/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:24:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 27 08:24:12.011320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474409e-0caf-4ab9-b315-04142f99839e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:12.105880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:12.112557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474409e-0caf-4ab9-b315-04142f99839e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:12.145033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:12.145659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:12.145987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:12.146235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:12.146860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:12.148219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 27 08:24:12.157352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 27 08:24:12.158368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.get_or_create( Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application with Lock( Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self._enter() Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application return self.creator() Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application created_value = creator( Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: ERROR keystone.server.flask.application Jul 27 08:24:12.159259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 658/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 27 08:24:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 27 08:24:12.164490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce0eaf-c72f-43d9-97ef-e3887e0da4c9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:12.238553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:12.259920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce0eaf-c72f-43d9-97ef-e3887e0da4c9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:12.293780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795490b5-78c3-46b3-be0e-8516bdac8e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:12.294412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795490b5-78c3-46b3-be0e-8516bdac8e01 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:12.294693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795490b5-78c3-46b3-be0e-8516bdac8e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:12.294963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795490b5-78c3-46b3-be0e-8516bdac8e01 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:12.295628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795490b5-78c3-46b3-be0e-8516bdac8e01 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:24:12.295932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795490b5-78c3-46b3-be0e-8516bdac8e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:12.297406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795490b5-78c3-46b3-be0e-8516bdac8e01 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:12.302409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 647/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 27 08:24:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:24:12.308245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:24:12.408300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c916b-1762-4b07-a55a-c405c5f0d86c None admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:24:12.442042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce2943-a84e-4a89-bba0-54068beba1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:24:12.442672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2943-a84e-4a89-bba0-54068beba1b6 None admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:24:12.442952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2943-a84e-4a89-bba0-54068beba1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:24:12.443227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce2943-a84e-4a89-bba0-54068beba1b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:24:12.443939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce2943-a84e-4a89-bba0-54068beba1b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:24:12.445385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce2943-a84e-4a89-bba0-54068beba1b6 None admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:24:12.531348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 659/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:24:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:24:22.299430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=30 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:24:22.526768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:18.999840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2ebd6-c271-4430-a138-7e7fd4645934 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:19.000767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2ebd6-c271-4430-a138-7e7fd4645934 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:19.001187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2ebd6-c271-4430-a138-7e7fd4645934 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:19.003012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 648/1307] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:19.008232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329d8a3-dc3e-4a98-ab08-93cd7423dbb7 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:19.009591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329d8a3-dc3e-4a98-ab08-93cd7423dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:19.010033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329d8a3-dc3e-4a98-ab08-93cd7423dbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:19.105650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b329d8a3-dc3e-4a98-ab08-93cd7423dbb7 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:19.107181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b329d8a3-dc3e-4a98-ab08-93cd7423dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:19.234156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 660/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:19.299066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04439-8890-44d0-939a-8beacf9c1ab4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:19.299646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04439-8890-44d0-939a-8beacf9c1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:19.300573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04439-8890-44d0-939a-8beacf9c1ab4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:19.303693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 649/1309] 10.222.0.40 () {54 vars in 965 bytes} [Wed Jul 27 08:30:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:19.319218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b4649-743c-49fa-916f-5086f1740eb3 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:19.320045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b4649-743c-49fa-916f-5086f1740eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:19.320533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b4649-743c-49fa-916f-5086f1740eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:19.478615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a85b4649-743c-49fa-916f-5086f1740eb3 None None] MFA Rules not processed for user `d5c308c33c674f1ea7111c0297174c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:19.480208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a85b4649-743c-49fa-916f-5086f1740eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:19.505561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:19.586306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 661/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Wed Jul 27 08:30:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:30:19.596148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04439-8890-44d0-939a-8beacf9c1ab4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:19.598351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f7c04439-8890-44d0-939a-8beacf9c1ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:19.822895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04439-8890-44d0-939a-8beacf9c1ab4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:19.966756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb7154-fd70-4773-ab7f-b1bbb2dd833b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:19.967463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb7154-fd70-4773-ab7f-b1bbb2dd833b service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:19.967694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb7154-fd70-4773-ab7f-b1bbb2dd833b service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:19.967903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb7154-fd70-4773-ab7f-b1bbb2dd833b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:19.969606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c2fb7154-fd70-4773-ab7f-b1bbb2dd833b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:20.035167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:20.158420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb7154-fd70-4773-ab7f-b1bbb2dd833b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:20.160402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb7154-fd70-4773-ab7f-b1bbb2dd833b service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:20.187785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 650/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:30:22.547323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbdba3-8075-4451-8b31-0584d3df0fbc None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:22.547888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbdba3-8075-4451-8b31-0584d3df0fbc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:22.547888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbdba3-8075-4451-8b31-0584d3df0fbc None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:22.549264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 662/1312] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:22.554573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62616d36-e0ab-4756-b76a-a0f0b041dc95 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:22.555371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62616d36-e0ab-4756-b76a-a0f0b041dc95 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:22.555654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62616d36-e0ab-4756-b76a-a0f0b041dc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:22.646438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-62616d36-e0ab-4756-b76a-a0f0b041dc95 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:22.647672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-62616d36-e0ab-4756-b76a-a0f0b041dc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:22.750542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 651/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:22.789819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbdba3-8075-4451-8b31-0584d3df0fbc None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:22.791392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-28bbdba3-8075-4451-8b31-0584d3df0fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:22.903169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbdba3-8075-4451-8b31-0584d3df0fbc None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:22.978408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:22.981041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0371dcc-f21a-45cc-bbd3-57d2a94921a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:22.981696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0371dcc-f21a-45cc-bbd3-57d2a94921a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:22.982924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0371dcc-f21a-45cc-bbd3-57d2a94921a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:22.983242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0371dcc-f21a-45cc-bbd3-57d2a94921a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:22.984909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c0371dcc-f21a-45cc-bbd3-57d2a94921a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:23.094571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0371dcc-f21a-45cc-bbd3-57d2a94921a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:23.096155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0371dcc-f21a-45cc-bbd3-57d2a94921a8 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:23.113683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 663/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:25.606954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ca2f1-4f4e-420f-ad12-8d0cb13d4db7 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:25.607452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ca2f1-4f4e-420f-ad12-8d0cb13d4db7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:25.607546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ca2f1-4f4e-420f-ad12-8d0cb13d4db7 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:25.608873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 652/1315] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:25.614014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ae551-95ef-4eca-a858-c4b8bf85bda2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:25.614449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ae551-95ef-4eca-a858-c4b8bf85bda2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:25.614584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ae551-95ef-4eca-a858-c4b8bf85bda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:25.679276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2f3ae551-95ef-4eca-a858-c4b8bf85bda2 None None] MFA Rules not processed for user `ac3564120871420cae8d959903f7be82`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:25.680459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2f3ae551-95ef-4eca-a858-c4b8bf85bda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:25.732261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:25.775661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 664/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:25.816388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ca2f1-4f4e-420f-ad12-8d0cb13d4db7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:25.817927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1b5ca2f1-4f4e-420f-ad12-8d0cb13d4db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:25.828227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:25.920741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ca2f1-4f4e-420f-ad12-8d0cb13d4db7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:25.992344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db7c71-8142-486a-b39b-e4c485ee3b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:25.992951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7c71-8142-486a-b39b-e4c485ee3b28 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:25.993226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7c71-8142-486a-b39b-e4c485ee3b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:25.993489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7c71-8142-486a-b39b-e4c485ee3b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:25.995104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-93db7c71-8142-486a-b39b-e4c485ee3b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:26.057263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:26.103690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db7c71-8142-486a-b39b-e4c485ee3b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:26.105055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db7c71-8142-486a-b39b-e4c485ee3b28 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:26.124468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 653/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:28.949641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de550e42-3dcf-490a-be32-1ef5e14f5145 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:28.950261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de550e42-3dcf-490a-be32-1ef5e14f5145 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:28.950415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de550e42-3dcf-490a-be32-1ef5e14f5145 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:28.951718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 665/1318] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:28.956854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cd76c-974e-44ae-ace7-0a2f256712ab None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:28.957276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cd76c-974e-44ae-ace7-0a2f256712ab None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:28.957426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cd76c-974e-44ae-ace7-0a2f256712ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:29.023105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8b2cd76c-974e-44ae-ace7-0a2f256712ab None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:29.024306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8b2cd76c-974e-44ae-ace7-0a2f256712ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:29.118171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 654/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:29.157620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de550e42-3dcf-490a-be32-1ef5e14f5145 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:29.159160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-de550e42-3dcf-490a-be32-1ef5e14f5145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:29.263535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de550e42-3dcf-490a-be32-1ef5e14f5145 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:29.334913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93889e5-c262-4c41-b051-bd90ecaa0495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:29.335525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93889e5-c262-4c41-b051-bd90ecaa0495 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:29.335879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93889e5-c262-4c41-b051-bd90ecaa0495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:29.336155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93889e5-c262-4c41-b051-bd90ecaa0495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:29.337764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a93889e5-c262-4c41-b051-bd90ecaa0495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:29.346173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:29.445069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93889e5-c262-4c41-b051-bd90ecaa0495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:29.446561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93889e5-c262-4c41-b051-bd90ecaa0495 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:29.464454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 666/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:32.453391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df7687-be0f-49b0-b77d-1773342ba865 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:32.453932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df7687-be0f-49b0-b77d-1773342ba865 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:32.453932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df7687-be0f-49b0-b77d-1773342ba865 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:32.456552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 655/1321] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:32.460496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2c6f7-fd35-4ff8-89de-204b6ffc9fa0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:32.460927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2c6f7-fd35-4ff8-89de-204b6ffc9fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:32.461022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2c6f7-fd35-4ff8-89de-204b6ffc9fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:32.536797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-7bc2c6f7-fd35-4ff8-89de-204b6ffc9fa0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:32.537984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7bc2c6f7-fd35-4ff8-89de-204b6ffc9fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:32.601541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:32.644055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 667/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:32.682258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df7687-be0f-49b0-b77d-1773342ba865 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:32.684860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-28df7687-be0f-49b0-b77d-1773342ba865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:32.712935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:32.811270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df7687-be0f-49b0-b77d-1773342ba865 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:32.897560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d5d65-c266-4bc3-89a4-5da73b09068c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:32.898423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d5d65-c266-4bc3-89a4-5da73b09068c service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:32.898706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d5d65-c266-4bc3-89a4-5da73b09068c service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:32.898980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d5d65-c266-4bc3-89a4-5da73b09068c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:32.900641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7f7d5d65-c266-4bc3-89a4-5da73b09068c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:32.985986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:33.025133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d5d65-c266-4bc3-89a4-5da73b09068c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:33.027073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d5d65-c266-4bc3-89a4-5da73b09068c service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:33.048146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 656/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:35.155974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbb0fe-e111-46c0-8acc-ded46c824f96 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:35.156575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbb0fe-e111-46c0-8acc-ded46c824f96 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:35.156575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbb0fe-e111-46c0-8acc-ded46c824f96 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:35.157888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 668/1324] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:35.163172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688e92-a092-4f37-82b0-17b8acd5384b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:35.163613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688e92-a092-4f37-82b0-17b8acd5384b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:35.163734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45688e92-a092-4f37-82b0-17b8acd5384b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:35.241573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-45688e92-a092-4f37-82b0-17b8acd5384b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:35.242799 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-45688e92-a092-4f37-82b0-17b8acd5384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:35.347337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 657/1325] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:35.407716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0efc9d-8e5a-4afc-8b38-3a07c4db200a None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:35.408044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0efc9d-8e5a-4afc-8b38-3a07c4db200a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:35.408265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0efc9d-8e5a-4afc-8b38-3a07c4db200a None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:35.409777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 669/1326] 10.222.0.40 () {54 vars in 965 bytes} [Wed Jul 27 08:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:35.415863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1084cb-ceb2-4b0b-9c99-b143ea9f4049 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:35.416238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1084cb-ceb2-4b0b-9c99-b143ea9f4049 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:35.416514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1084cb-ceb2-4b0b-9c99-b143ea9f4049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:35.448588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:35.495959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4f1084cb-ceb2-4b0b-9c99-b143ea9f4049 None None] MFA Rules not processed for user `d5c308c33c674f1ea7111c0297174c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:35.498070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4f1084cb-ceb2-4b0b-9c99-b143ea9f4049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:35.581951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 658/1327] 10.222.0.40 () {58 vars in 1062 bytes} [Wed Jul 27 08:30:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:35.592312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0efc9d-8e5a-4afc-8b38-3a07c4db200a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:35.593848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8a0efc9d-8e5a-4afc-8b38-3a07c4db200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:35.719054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0efc9d-8e5a-4afc-8b38-3a07c4db200a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:35.799092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec681e-c5c7-443b-975d-3fb0081a8be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:35.800059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec681e-c5c7-443b-975d-3fb0081a8be3 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:35.800495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec681e-c5c7-443b-975d-3fb0081a8be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:35.800993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec681e-c5c7-443b-975d-3fb0081a8be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:35.802887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-efec681e-c5c7-443b-975d-3fb0081a8be3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:35.822473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:35.927338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec681e-c5c7-443b-975d-3fb0081a8be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:35.929044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec681e-c5c7-443b-975d-3fb0081a8be3 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:35.950460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 670/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:38.251741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e5e0-bba8-43a5-921f-3dc8a3b4b16a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:38.252288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e5e0-bba8-43a5-921f-3dc8a3b4b16a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:38.252288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e5e0-bba8-43a5-921f-3dc8a3b4b16a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:38.254484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 659/1329] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:38.258817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47634-e2b9-4c02-be3d-72cb51a5694e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:38.259334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47634-e2b9-4c02-be3d-72cb51a5694e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:38.259418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47634-e2b9-4c02-be3d-72cb51a5694e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:38.337868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f0d47634-e2b9-4c02-be3d-72cb51a5694e None None] MFA Rules not processed for user `ac3564120871420cae8d959903f7be82`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:38.339292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f0d47634-e2b9-4c02-be3d-72cb51a5694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:38.411171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:38.466694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 671/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:38.489992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5e5e0-bba8-43a5-921f-3dc8a3b4b16a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:38.491572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-aed5e5e0-bba8-43a5-921f-3dc8a3b4b16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:38.628641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5e5e0-bba8-43a5-921f-3dc8a3b4b16a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:38.659585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:38.720325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca6b46-1ff8-4d56-bf64-608fb5820d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:38.720940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca6b46-1ff8-4d56-bf64-608fb5820d70 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:38.721217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca6b46-1ff8-4d56-bf64-608fb5820d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:38.721487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca6b46-1ff8-4d56-bf64-608fb5820d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:38.723174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-26ca6b46-1ff8-4d56-bf64-608fb5820d70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:38.840457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca6b46-1ff8-4d56-bf64-608fb5820d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:38.849214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca6b46-1ff8-4d56-bf64-608fb5820d70 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:38.862255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 660/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:41.513572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d89e1-4651-4af7-b224-26d04be48cc8 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:41.514113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d89e1-4651-4af7-b224-26d04be48cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:41.514113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d89e1-4651-4af7-b224-26d04be48cc8 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:41.515491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 672/1332] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:41.520739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89ab5-3dd5-46d6-8c2c-b596df31736e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:41.521157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89ab5-3dd5-46d6-8c2c-b596df31736e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:41.521292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89ab5-3dd5-46d6-8c2c-b596df31736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:41.584848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:41.587701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-adc89ab5-3dd5-46d6-8c2c-b596df31736e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:41.588921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-adc89ab5-3dd5-46d6-8c2c-b596df31736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:41.684934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 661/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:41.721285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d89e1-4651-4af7-b224-26d04be48cc8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:41.722850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dc2d89e1-4651-4af7-b224-26d04be48cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:41.823436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d89e1-4651-4af7-b224-26d04be48cc8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:41.894762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466612a-5c5e-4f61-8c27-0d730721cb70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:41.895368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466612a-5c5e-4f61-8c27-0d730721cb70 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:41.895645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466612a-5c5e-4f61-8c27-0d730721cb70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:41.895914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466612a-5c5e-4f61-8c27-0d730721cb70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:41.897491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4466612a-5c5e-4f61-8c27-0d730721cb70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:41.923909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:42.007054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466612a-5c5e-4f61-8c27-0d730721cb70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:42.008488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466612a-5c5e-4f61-8c27-0d730721cb70 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:42.025988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 673/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:44.051260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc2441-75fd-4d62-b66f-4fa51e2bb8a0 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:44.051861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc2441-75fd-4d62-b66f-4fa51e2bb8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:44.051861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc2441-75fd-4d62-b66f-4fa51e2bb8a0 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:44.053239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 662/1335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:44.058495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9f33c-df6e-4b01-b830-f1fd6378c041 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:44.059440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9f33c-df6e-4b01-b830-f1fd6378c041 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:44.059636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9f33c-df6e-4b01-b830-f1fd6378c041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:44.125730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2ee9f33c-df6e-4b01-b830-f1fd6378c041 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:44.126974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2ee9f33c-df6e-4b01-b830-f1fd6378c041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:44.199262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:44.221572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 674/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:44.261565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc2441-75fd-4d62-b66f-4fa51e2bb8a0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:44.263140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9bfc2441-75fd-4d62-b66f-4fa51e2bb8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:44.364350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc2441-75fd-4d62-b66f-4fa51e2bb8a0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:44.391878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:44.435869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbbbe8-40a7-4b1c-b012-1d8abdc35cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:44.436475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbbbe8-40a7-4b1c-b012-1d8abdc35cac service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:44.436753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbbbe8-40a7-4b1c-b012-1d8abdc35cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:44.437025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbbbe8-40a7-4b1c-b012-1d8abdc35cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:44.438656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-39dbbbe8-40a7-4b1c-b012-1d8abdc35cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:44.546241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbbbe8-40a7-4b1c-b012-1d8abdc35cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:44.547682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbbbe8-40a7-4b1c-b012-1d8abdc35cac service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:44.568422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 663/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:47.284886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a9ec5-e2b1-4eb9-b2b6-6d38b8d75326 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:47.285514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a9ec5-e2b1-4eb9-b2b6-6d38b8d75326 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:47.285514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a9ec5-e2b1-4eb9-b2b6-6d38b8d75326 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:47.286760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 675/1338] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:47.291783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57f6c-d04b-4f2a-ab04-77dea60bfaeb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:47.292201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57f6c-d04b-4f2a-ab04-77dea60bfaeb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:47.292311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf57f6c-d04b-4f2a-ab04-77dea60bfaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:47.357586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5cf57f6c-d04b-4f2a-ab04-77dea60bfaeb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:47.358806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5cf57f6c-d04b-4f2a-ab04-77dea60bfaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:47.366811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:47.451857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 664/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:47.489296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a9ec5-e2b1-4eb9-b2b6-6d38b8d75326 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:47.490907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-592a9ec5-e2b1-4eb9-b2b6-6d38b8d75326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:47.592182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a9ec5-e2b1-4eb9-b2b6-6d38b8d75326 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:47.665287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d120b77-e095-45cf-8849-4e09cd726121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:47.665900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d120b77-e095-45cf-8849-4e09cd726121 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:47.666203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d120b77-e095-45cf-8849-4e09cd726121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:47.666477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d120b77-e095-45cf-8849-4e09cd726121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:47.668683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2d120b77-e095-45cf-8849-4e09cd726121 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:47.702856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:47.773277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d120b77-e095-45cf-8849-4e09cd726121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:47.774733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d120b77-e095-45cf-8849-4e09cd726121 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:47.793589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 676/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:30:52.055277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c935362-cc6d-436f-82a4-a6c798958a8a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:52.058557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c935362-cc6d-436f-82a4-a6c798958a8a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:52.058557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c935362-cc6d-436f-82a4-a6c798958a8a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:52.058557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 665/1341] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:52.062542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bece4a-e5b2-442d-bee1-56ecc39a66e7 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:52.062942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bece4a-e5b2-442d-bee1-56ecc39a66e7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:52.063146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bece4a-e5b2-442d-bee1-56ecc39a66e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:52.135562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-97bece4a-e5b2-442d-bee1-56ecc39a66e7 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:52.137133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-97bece4a-e5b2-442d-bee1-56ecc39a66e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:52.227674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:52.239439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 677/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:52.278919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c935362-cc6d-436f-82a4-a6c798958a8a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:52.280512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3c935362-cc6d-436f-82a4-a6c798958a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:52.384800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c935362-cc6d-436f-82a4-a6c798958a8a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:52.423233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:52.459401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba3adf-a650-4caf-b4e4-01c30431ac5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:52.460012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3adf-a650-4caf-b4e4-01c30431ac5c service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:52.460285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3adf-a650-4caf-b4e4-01c30431ac5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:52.460553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba3adf-a650-4caf-b4e4-01c30431ac5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:52.462931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6fba3adf-a650-4caf-b4e4-01c30431ac5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:52.575878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba3adf-a650-4caf-b4e4-01c30431ac5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:52.577317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba3adf-a650-4caf-b4e4-01c30431ac5c service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:52.597691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 666/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:30:55.560510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afca29e-1780-4384-be41-ffbdcedc8430 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:55.561033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afca29e-1780-4384-be41-ffbdcedc8430 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:55.561033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afca29e-1780-4384-be41-ffbdcedc8430 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:55.562666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 678/1344] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:30:55.567665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15e1ae-4bc9-4e32-b54f-b00d5a2ce2cc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:55.568172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15e1ae-4bc9-4e32-b54f-b00d5a2ce2cc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:55.568268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15e1ae-4bc9-4e32-b54f-b00d5a2ce2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:55.635149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cc15e1ae-4bc9-4e32-b54f-b00d5a2ce2cc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:30:55.636927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cc15e1ae-4bc9-4e32-b54f-b00d5a2ce2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:55.655316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:55.736711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 667/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:30:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:30:55.775279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afca29e-1780-4384-be41-ffbdcedc8430 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:30:55.776879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2afca29e-1780-4384-be41-ffbdcedc8430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:55.879236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afca29e-1780-4384-be41-ffbdcedc8430 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:30:55.951861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fb11d-f24b-43e8-823f-938f7fef2723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:30:55.952469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fb11d-f24b-43e8-823f-938f7fef2723 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:30:55.952744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fb11d-f24b-43e8-823f-938f7fef2723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:30:55.953013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fb11d-f24b-43e8-823f-938f7fef2723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:30:55.954636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7c6fb11d-f24b-43e8-823f-938f7fef2723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:30:55.998538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:30:56.065549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fb11d-f24b-43e8-823f-938f7fef2723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:30:56.067043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fb11d-f24b-43e8-823f-938f7fef2723 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:30:56.088475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 679/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:30:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:31:00.249049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548504b-c4a9-4ea8-9239-7270482b9371 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:00.249553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548504b-c4a9-4ea8-9239-7270482b9371 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:00.249673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548504b-c4a9-4ea8-9239-7270482b9371 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:00.251152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 668/1347] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:00.256376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc6e6f-53e7-462e-8690-e44c6680801a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:00.256838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc6e6f-53e7-462e-8690-e44c6680801a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:00.256928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc6e6f-53e7-462e-8690-e44c6680801a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:00.328550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fabc6e6f-53e7-462e-8690-e44c6680801a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:00.329775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fabc6e6f-53e7-462e-8690-e44c6680801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:00.430091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:00.432800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 680/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:00.472834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548504b-c4a9-4ea8-9239-7270482b9371 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:00.474421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4548504b-c4a9-4ea8-9239-7270482b9371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:00.585432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548504b-c4a9-4ea8-9239-7270482b9371 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:00.638629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:00.666101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8ae2-71ee-4253-b8b8-aea35c8db327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:00.666806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8ae2-71ee-4253-b8b8-aea35c8db327 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:00.667121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8ae2-71ee-4253-b8b8-aea35c8db327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:00.667406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8ae2-71ee-4253-b8b8-aea35c8db327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:00.669066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2edf8ae2-71ee-4253-b8b8-aea35c8db327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:00.781120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8ae2-71ee-4253-b8b8-aea35c8db327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:00.782614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8ae2-71ee-4253-b8b8-aea35c8db327 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:00.802753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 669/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:05.898153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4453b7-5c68-49a2-b104-88323ef85227 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:05.898634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4453b7-5c68-49a2-b104-88323ef85227 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:05.898727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4453b7-5c68-49a2-b104-88323ef85227 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:05.900331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 681/1350] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:31:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:05.906241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4330f-3a55-44b6-b380-7b1f38d9dff4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:05.906659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4330f-3a55-44b6-b380-7b1f38d9dff4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:05.906841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4330f-3a55-44b6-b380-7b1f38d9dff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:06.020534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-62f4330f-3a55-44b6-b380-7b1f38d9dff4 None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:06.020534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-62f4330f-3a55-44b6-b380-7b1f38d9dff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:06.053222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:06.204531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 670/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:31:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:10.355748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdea8f-4941-4333-b997-81df4bf47287 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:10.356312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdea8f-4941-4333-b997-81df4bf47287 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:10.356312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdea8f-4941-4333-b997-81df4bf47287 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:10.357827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 682/1352] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:10.362845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699fd3a-70ab-4231-bca7-474995ccda6d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:10.363239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699fd3a-70ab-4231-bca7-474995ccda6d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:10.363402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699fd3a-70ab-4231-bca7-474995ccda6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:10.444071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2699fd3a-70ab-4231-bca7-474995ccda6d None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:10.445669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2699fd3a-70ab-4231-bca7-474995ccda6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:10.522809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:10.565492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 671/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:31:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:11.876814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df5cb-c1a5-42e8-80a5-1e2d7b121da0 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:11.877379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df5cb-c1a5-42e8-80a5-1e2d7b121da0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:11.877379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8df5cb-c1a5-42e8-80a5-1e2d7b121da0 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:11.878860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 683/1354] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:11.884356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815bb32-4d5f-4c22-bac8-fdf08b859d66 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:11.884790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815bb32-4d5f-4c22-bac8-fdf08b859d66 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:11.884951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815bb32-4d5f-4c22-bac8-fdf08b859d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:11.964334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4815bb32-4d5f-4c22-bac8-fdf08b859d66 None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:11.965916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4815bb32-4d5f-4c22-bac8-fdf08b859d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:12.074736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 672/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:31:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:15.476440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d160e19-eabe-43ba-a128-ad3bd7ceae9d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:15.476959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d160e19-eabe-43ba-a128-ad3bd7ceae9d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:15.477040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d160e19-eabe-43ba-a128-ad3bd7ceae9d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:15.478739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 684/1356] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:15.484938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442ebf-73e7-4e51-bbee-73b80083df7d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:15.485417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442ebf-73e7-4e51-bbee-73b80083df7d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:15.485511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442ebf-73e7-4e51-bbee-73b80083df7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:15.527425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:15.565320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-21442ebf-73e7-4e51-bbee-73b80083df7d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:15.566812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-21442ebf-73e7-4e51-bbee-73b80083df7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:15.679552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 673/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:15.736368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e716dd-edf6-43dc-b7a1-6bdc7ad2ff61 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:15.737414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e716dd-edf6-43dc-b7a1-6bdc7ad2ff61 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:15.738177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e716dd-edf6-43dc-b7a1-6bdc7ad2ff61 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:15.739709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 685/1358] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 27 08:31:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:15.747242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b01ee8-42d6-4097-8c8d-4caf2b196760 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:15.747529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b01ee8-42d6-4097-8c8d-4caf2b196760 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:15.752666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b01ee8-42d6-4097-8c8d-4caf2b196760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:15.833569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-46b01ee8-42d6-4097-8c8d-4caf2b196760 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:15.834796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-46b01ee8-42d6-4097-8c8d-4caf2b196760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:15.886441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:15.968494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 674/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 27 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:31:15.977700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e716dd-edf6-43dc-b7a1-6bdc7ad2ff61 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:15.978787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e0e716dd-edf6-43dc-b7a1-6bdc7ad2ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:16.130254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e716dd-edf6-43dc-b7a1-6bdc7ad2ff61 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:16.209459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:16.251177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a400f9-a991-4c93-b5a6-c8db650f6440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:16.251804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a400f9-a991-4c93-b5a6-c8db650f6440 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:16.252083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a400f9-a991-4c93-b5a6-c8db650f6440 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:16.252355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a400f9-a991-4c93-b5a6-c8db650f6440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:16.253957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-20a400f9-a991-4c93-b5a6-c8db650f6440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:16.348546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a400f9-a991-4c93-b5a6-c8db650f6440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:16.350192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a400f9-a991-4c93-b5a6-c8db650f6440 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:16.370014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 686/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:18.641643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7a2ec-6e8e-408b-96aa-aa789efbac54 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:18.642391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7a2ec-6e8e-408b-96aa-aa789efbac54 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:18.642391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7a2ec-6e8e-408b-96aa-aa789efbac54 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:18.643761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 675/1361] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:18.648714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e7687-6aea-46f3-a47f-ac1a21d537fc None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:18.649136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e7687-6aea-46f3-a47f-ac1a21d537fc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:18.649220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e7687-6aea-46f3-a47f-ac1a21d537fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:18.723614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ba1e7687-6aea-46f3-a47f-ac1a21d537fc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:18.724815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ba1e7687-6aea-46f3-a47f-ac1a21d537fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:18.766343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:18.827539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 687/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:18.872705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7a2ec-6e8e-408b-96aa-aa789efbac54 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:18.874418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-38c7a2ec-6e8e-408b-96aa-aa789efbac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:19.062445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7a2ec-6e8e-408b-96aa-aa789efbac54 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:19.073579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:19.198988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740e9da-7567-4410-95f7-df7d1741a21a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:19.202976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e9da-7567-4410-95f7-df7d1741a21a service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:19.203295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e9da-7567-4410-95f7-df7d1741a21a service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:19.203295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740e9da-7567-4410-95f7-df7d1741a21a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:19.205195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e740e9da-7567-4410-95f7-df7d1741a21a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:19.302648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740e9da-7567-4410-95f7-df7d1741a21a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:19.304254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740e9da-7567-4410-95f7-df7d1741a21a service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:19.324985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 676/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:21.905081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a212a17-db3b-4376-ba3f-8757f1520271 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:21.905629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a212a17-db3b-4376-ba3f-8757f1520271 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:21.905629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a212a17-db3b-4376-ba3f-8757f1520271 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:21.907041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 688/1364] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:21.912529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2a27-a511-4ae8-817b-8d0041f1aec4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:21.913003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2a27-a511-4ae8-817b-8d0041f1aec4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:21.913104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb2a27-a511-4ae8-817b-8d0041f1aec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:21.930043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:21.986480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2abb2a27-a511-4ae8-817b-8d0041f1aec4 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:21.987718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2abb2a27-a511-4ae8-817b-8d0041f1aec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:22.091669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 677/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:22.138581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda9cdb-629d-4a48-a10e-8a777181c210 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:22.138915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda9cdb-629d-4a48-a10e-8a777181c210 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:22.139687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda9cdb-629d-4a48-a10e-8a777181c210 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:22.141217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 689/1366] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 27 08:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:22.147006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc1d3b-8bfd-4a2b-9f05-6d7c2890d51c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:22.147289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc1d3b-8bfd-4a2b-9f05-6d7c2890d51c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:22.148882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc1d3b-8bfd-4a2b-9f05-6d7c2890d51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:22.246778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a0fc1d3b-8bfd-4a2b-9f05-6d7c2890d51c None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:22.250288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a0fc1d3b-8bfd-4a2b-9f05-6d7c2890d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:22.288765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:22.386404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 678/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 27 08:31:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:22.395020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda9cdb-629d-4a48-a10e-8a777181c210 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:22.396823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8dda9cdb-629d-4a48-a10e-8a777181c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:22.549623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda9cdb-629d-4a48-a10e-8a777181c210 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:22.575841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:22.678261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3bf6cc-5055-4783-a946-f4eb8f7d3090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:22.678869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf6cc-5055-4783-a946-f4eb8f7d3090 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:22.679144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf6cc-5055-4783-a946-f4eb8f7d3090 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:22.679411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3bf6cc-5055-4783-a946-f4eb8f7d3090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:22.681001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8c3bf6cc-5055-4783-a946-f4eb8f7d3090 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:22.779291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bf6cc-5055-4783-a946-f4eb8f7d3090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:22.780898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3bf6cc-5055-4783-a946-f4eb8f7d3090 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:22.800283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 690/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:31:23.668235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc856e-2c9e-49c1-b09a-f172d59775c3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:23.668766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc856e-2c9e-49c1-b09a-f172d59775c3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:23.668766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc856e-2c9e-49c1-b09a-f172d59775c3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:23.670198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 679/1369] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:23.676057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30128a-6218-4524-ad31-d9fa78967bd1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:23.676551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30128a-6218-4524-ad31-d9fa78967bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:23.676633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30128a-6218-4524-ad31-d9fa78967bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:23.711490 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:23.749249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4a30128a-6218-4524-ad31-d9fa78967bd1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:23.750570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4a30128a-6218-4524-ad31-d9fa78967bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:23.853117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 691/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:24.175033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbbdec-37a9-4933-859f-4fa44d9630c9 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:24.175517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbbdec-37a9-4933-859f-4fa44d9630c9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:24.175802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbbdec-37a9-4933-859f-4fa44d9630c9 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:24.177465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 680/1371] 10.222.0.40 () {54 vars in 964 bytes} [Wed Jul 27 08:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:24.183331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24751077-a800-4337-a9f7-875a4da3254c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:24.183615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24751077-a800-4337-a9f7-875a4da3254c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:24.183892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24751077-a800-4337-a9f7-875a4da3254c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:24.267141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-24751077-a800-4337-a9f7-875a4da3254c None None] MFA Rules not processed for user `99a29a23f8a84076bab3adefc042a088`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:24.268665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-24751077-a800-4337-a9f7-875a4da3254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:24.325367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:24.359382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 692/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Jul 27 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:24.366833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbbdec-37a9-4933-859f-4fa44d9630c9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:24.368329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8ecbbdec-37a9-4933-859f-4fa44d9630c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:24.475305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbbdec-37a9-4933-859f-4fa44d9630c9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:24.519706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:24.559862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9a514-9498-4f68-b0e2-b8f37192f3dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:24.561094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a514-9498-4f68-b0e2-b8f37192f3dd service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:24.561440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a514-9498-4f68-b0e2-b8f37192f3dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:24.561801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a514-9498-4f68-b0e2-b8f37192f3dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:24.563517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-afe9a514-9498-4f68-b0e2-b8f37192f3dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:24.682292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9a514-9498-4f68-b0e2-b8f37192f3dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:24.683745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9a514-9498-4f68-b0e2-b8f37192f3dd service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:24.704478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 681/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:25.004924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b9f79-a0b5-4bdd-9cc5-65841bf09e66 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:25.005406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b9f79-a0b5-4bdd-9cc5-65841bf09e66 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:25.005710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b9f79-a0b5-4bdd-9cc5-65841bf09e66 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:25.010934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 693/1374] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:25.012935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aebfe6-7f9f-4050-9310-49643ce46780 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:25.013202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aebfe6-7f9f-4050-9310-49643ce46780 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:25.013473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aebfe6-7f9f-4050-9310-49643ce46780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:25.090774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-42aebfe6-7f9f-4050-9310-49643ce46780 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:25.091842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-42aebfe6-7f9f-4050-9310-49643ce46780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:25.114741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:25.199880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 682/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:25.244510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b9f79-a0b5-4bdd-9cc5-65841bf09e66 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:25.246130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-620b9f79-a0b5-4bdd-9cc5-65841bf09e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:25.405812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b9f79-a0b5-4bdd-9cc5-65841bf09e66 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:25.456147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:25.534624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286db1e-38fd-4ca6-a0c7-c49599a9daf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:25.535233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286db1e-38fd-4ca6-a0c7-c49599a9daf7 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:25.535508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286db1e-38fd-4ca6-a0c7-c49599a9daf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:25.535776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286db1e-38fd-4ca6-a0c7-c49599a9daf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:25.537368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4286db1e-38fd-4ca6-a0c7-c49599a9daf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:25.630335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286db1e-38fd-4ca6-a0c7-c49599a9daf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:25.631970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286db1e-38fd-4ca6-a0c7-c49599a9daf7 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:25.650929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 694/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:27.796847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a6f1-ae37-4008-a052-3597fcb6ed21 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:27.797268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a6f1-ae37-4008-a052-3597fcb6ed21 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:27.797400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a6f1-ae37-4008-a052-3597fcb6ed21 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:27.798798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 683/1377] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:27.804050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80e91d-5a00-4a65-9d99-62d6e9df9e0c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:27.804494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80e91d-5a00-4a65-9d99-62d6e9df9e0c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:27.804620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80e91d-5a00-4a65-9d99-62d6e9df9e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:27.888707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-da80e91d-5a00-4a65-9d99-62d6e9df9e0c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:27.889914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-da80e91d-5a00-4a65-9d99-62d6e9df9e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:27.896806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:27.985896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 695/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:28.030281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a6f1-ae37-4008-a052-3597fcb6ed21 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:28.031906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b514a6f1-ae37-4008-a052-3597fcb6ed21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:28.181775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a6f1-ae37-4008-a052-3597fcb6ed21 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:28.185441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:28.326334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70441585-ec9d-47be-923e-7ff26f134b22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:28.327209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70441585-ec9d-47be-923e-7ff26f134b22 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:28.327473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70441585-ec9d-47be-923e-7ff26f134b22 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:28.327813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70441585-ec9d-47be-923e-7ff26f134b22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:28.329532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-70441585-ec9d-47be-923e-7ff26f134b22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:28.430174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70441585-ec9d-47be-923e-7ff26f134b22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:28.431897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70441585-ec9d-47be-923e-7ff26f134b22 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:28.451372 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 684/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:31:30.624087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd964f-5ff2-45e5-815b-9665fdb525dc None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:30.624566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd964f-5ff2-45e5-815b-9665fdb525dc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:30.624644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd964f-5ff2-45e5-815b-9665fdb525dc None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:30.626208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 696/1380] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:30.631616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26273b6c-284e-411c-8315-6a138e464911 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:30.632021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26273b6c-284e-411c-8315-6a138e464911 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:30.632210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26273b6c-284e-411c-8315-6a138e464911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:30.643827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:30.705944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-26273b6c-284e-411c-8315-6a138e464911 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:30.707165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-26273b6c-284e-411c-8315-6a138e464911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:30.811880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 685/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:30.855312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd964f-5ff2-45e5-815b-9665fdb525dc None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:30.856809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-31fd964f-5ff2-45e5-815b-9665fdb525dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:30.992248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:31.003384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd964f-5ff2-45e5-815b-9665fdb525dc None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:31.121579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0bc82-caf1-4de9-8266-9f397b4cd6d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:31.122126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0bc82-caf1-4de9-8266-9f397b4cd6d5 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:31.122426 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0bc82-caf1-4de9-8266-9f397b4cd6d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:31.122697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0bc82-caf1-4de9-8266-9f397b4cd6d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:31.124434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7aa0bc82-caf1-4de9-8266-9f397b4cd6d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:31.232462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0bc82-caf1-4de9-8266-9f397b4cd6d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:31.234065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0bc82-caf1-4de9-8266-9f397b4cd6d5 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:31.246663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:31.256162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 697/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:33.405392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07827c-b2f8-411d-9e3f-83e136892c73 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:33.405935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07827c-b2f8-411d-9e3f-83e136892c73 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:33.406424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07827c-b2f8-411d-9e3f-83e136892c73 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:33.408200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 686/1383] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:33.414066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9936-ddc8-440f-b943-740b7066e2cb None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:33.414539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9936-ddc8-440f-b943-740b7066e2cb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:33.414791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9936-ddc8-440f-b943-740b7066e2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:33.485198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e75e9936-ddc8-440f-b943-740b7066e2cb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:33.486437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e75e9936-ddc8-440f-b943-740b7066e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:33.587471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 698/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:33.632439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07827c-b2f8-411d-9e3f-83e136892c73 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:33.634753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5c07827c-b2f8-411d-9e3f-83e136892c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:33.722023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:33.782733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07827c-b2f8-411d-9e3f-83e136892c73 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:33.899356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78680-f208-4581-aba0-9cf91931421c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:33.900219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78680-f208-4581-aba0-9cf91931421c service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:33.900564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78680-f208-4581-aba0-9cf91931421c service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:33.900906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78680-f208-4581-aba0-9cf91931421c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:33.902835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-67e78680-f208-4581-aba0-9cf91931421c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:33.957647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:33.999614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78680-f208-4581-aba0-9cf91931421c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:34.001146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78680-f208-4581-aba0-9cf91931421c service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:34.019972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 687/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:36.105184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6a16e-1dc1-44ec-929f-9b9a83720151 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:36.105765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6a16e-1dc1-44ec-929f-9b9a83720151 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:36.105765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6a16e-1dc1-44ec-929f-9b9a83720151 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:36.107196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 699/1386] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:36.113045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a99b0-d5e9-45eb-b943-5d393c5a3ab6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:36.113505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a99b0-d5e9-45eb-b943-5d393c5a3ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:36.113952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a99b0-d5e9-45eb-b943-5d393c5a3ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:36.190851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b11a99b0-d5e9-45eb-b943-5d393c5a3ab6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:36.192394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b11a99b0-d5e9-45eb-b943-5d393c5a3ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:36.310246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 688/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:36.357050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6a16e-1dc1-44ec-929f-9b9a83720151 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:36.358608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-94d6a16e-1dc1-44ec-929f-9b9a83720151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:36.437806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:36.507173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6a16e-1dc1-44ec-929f-9b9a83720151 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:36.623961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caf2e0-6451-4f41-b183-ca3823e00297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:36.624565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf2e0-6451-4f41-b183-ca3823e00297 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:36.624905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf2e0-6451-4f41-b183-ca3823e00297 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:36.625176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caf2e0-6451-4f41-b183-ca3823e00297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:36.626875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f1caf2e0-6451-4f41-b183-ca3823e00297 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:36.668749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:36.720659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caf2e0-6451-4f41-b183-ca3823e00297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:36.722289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1caf2e0-6451-4f41-b183-ca3823e00297 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:36.741872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 700/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:38.893328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d36ec-563d-4fa1-a1d3-5296c378b1ef None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:38.893890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d36ec-563d-4fa1-a1d3-5296c378b1ef None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:38.893890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d36ec-563d-4fa1-a1d3-5296c378b1ef None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:38.895474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 689/1389] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:38.901408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775779d-a769-4f02-b69d-84b29643c290 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:38.901811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775779d-a769-4f02-b69d-84b29643c290 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:38.902127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775779d-a769-4f02-b69d-84b29643c290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:38.980353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6775779d-a769-4f02-b69d-84b29643c290 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:38.981793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6775779d-a769-4f02-b69d-84b29643c290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:39.082276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 701/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:39.127370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d36ec-563d-4fa1-a1d3-5296c378b1ef None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:39.128905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d44d36ec-563d-4fa1-a1d3-5296c378b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:39.161742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:39.280979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d36ec-563d-4fa1-a1d3-5296c378b1ef None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:39.386604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:39.398322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703a0fa-e23f-4aad-a960-4c22d5320b26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:39.398943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703a0fa-e23f-4aad-a960-4c22d5320b26 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:39.399312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703a0fa-e23f-4aad-a960-4c22d5320b26 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:39.399586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703a0fa-e23f-4aad-a960-4c22d5320b26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:39.401217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8703a0fa-e23f-4aad-a960-4c22d5320b26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:39.500768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703a0fa-e23f-4aad-a960-4c22d5320b26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:39.502363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703a0fa-e23f-4aad-a960-4c22d5320b26 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:39.522768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 690/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:41.829335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ada96-b77c-4ae9-81ad-77971f28c4a8 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:41.829335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ada96-b77c-4ae9-81ad-77971f28c4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:41.829335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ada96-b77c-4ae9-81ad-77971f28c4a8 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:41.829335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 702/1392] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:41.830770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7ba5a-ed39-4b84-a027-1f52bad459c2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:41.831147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7ba5a-ed39-4b84-a027-1f52bad459c2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:41.831324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7ba5a-ed39-4b84-a027-1f52bad459c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:41.901528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3df7ba5a-ed39-4b84-a027-1f52bad459c2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:41.902756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3df7ba5a-ed39-4b84-a027-1f52bad459c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:41.961847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:42.001637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 691/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:42.046792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ada96-b77c-4ae9-81ad-77971f28c4a8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:42.048472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7c8ada96-b77c-4ae9-81ad-77971f28c4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:42.063189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:42.195234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ada96-b77c-4ae9-81ad-77971f28c4a8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:42.284131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:42.310134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8dee2-1f7d-4427-b70f-ad2d36e5062d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:42.310772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8dee2-1f7d-4427-b70f-ad2d36e5062d service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:42.311052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8dee2-1f7d-4427-b70f-ad2d36e5062d service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:42.311324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8dee2-1f7d-4427-b70f-ad2d36e5062d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:42.312967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-90d8dee2-1f7d-4427-b70f-ad2d36e5062d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:42.402510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8dee2-1f7d-4427-b70f-ad2d36e5062d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:42.404101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8dee2-1f7d-4427-b70f-ad2d36e5062d service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:42.422284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 703/1394] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:42.780113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecbdec-725c-4986-89a5-560aac059505 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:42.780300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecbdec-725c-4986-89a5-560aac059505 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:42.780686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecbdec-725c-4986-89a5-560aac059505 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:42.782064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 692/1395] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:42.787469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ad2e-306e-4e95-af70-9dae7321d925 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:42.792143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ad2e-306e-4e95-af70-9dae7321d925 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:42.792143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ad2e-306e-4e95-af70-9dae7321d925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:42.855761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3437ad2e-306e-4e95-af70-9dae7321d925 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:42.856960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3437ad2e-306e-4e95-af70-9dae7321d925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:42.908712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:42.956340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 704/1396] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:43.013866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a769cbf-7eac-4677-85c7-826e2cdf3d4f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:43.014420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a769cbf-7eac-4677-85c7-826e2cdf3d4f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:43.014531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a769cbf-7eac-4677-85c7-826e2cdf3d4f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:43.015827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 693/1397] 10.222.0.40 () {54 vars in 966 bytes} [Wed Jul 27 08:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:43.021312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f668-a07f-41ee-a9cd-9d2d6704e8ce None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:43.021669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f668-a07f-41ee-a9cd-9d2d6704e8ce None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:43.022012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9f668-a07f-41ee-a9cd-9d2d6704e8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:43.100363 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-feb9f668-a07f-41ee-a9cd-9d2d6704e8ce None None] MFA Rules not processed for user `4e41212825ad4990958a7e909c669177`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:43.101581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-feb9f668-a07f-41ee-a9cd-9d2d6704e8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:43.179678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 705/1398] 10.222.0.40 () {58 vars in 1063 bytes} [Wed Jul 27 08:31:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:43.187762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a769cbf-7eac-4677-85c7-826e2cdf3d4f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:43.189259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0a769cbf-7eac-4677-85c7-826e2cdf3d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:43.290538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a769cbf-7eac-4677-85c7-826e2cdf3d4f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:43.363837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d62d33-eba3-43b3-b0ed-5ff6698d72d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:43.364375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62d33-eba3-43b3-b0ed-5ff6698d72d6 service glance] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:43.364655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62d33-eba3-43b3-b0ed-5ff6698d72d6 service glance] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:43.364918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62d33-eba3-43b3-b0ed-5ff6698d72d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:43.366541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-12d62d33-eba3-43b3-b0ed-5ff6698d72d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:43.384896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:43.476252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d62d33-eba3-43b3-b0ed-5ff6698d72d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:43.478316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d62d33-eba3-43b3-b0ed-5ff6698d72d6 service glance] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:43.497422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 694/1399] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:44.346571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b9daa-273e-4bf8-8aaf-e4e48dde79e9 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:44.347182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b9daa-273e-4bf8-8aaf-e4e48dde79e9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:44.347182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b9daa-273e-4bf8-8aaf-e4e48dde79e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:44.407173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:44.416313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-567b9daa-273e-4bf8-8aaf-e4e48dde79e9 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:44.418602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-567b9daa-273e-4bf8-8aaf-e4e48dde79e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:44.506495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 706/1400] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:31:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:44.551920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0246ce-33fd-463f-a02a-867eda008204 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:44.551920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0246ce-33fd-463f-a02a-867eda008204 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:44.551920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0246ce-33fd-463f-a02a-867eda008204 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:44.552976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 695/1401] 10.222.0.40 () {54 vars in 943 bytes} [Wed Jul 27 08:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:44.559372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae50aa5-1638-472e-ae60-5ebd4db79cbe None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:44.559662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae50aa5-1638-472e-ae60-5ebd4db79cbe None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:44.559938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae50aa5-1638-472e-ae60-5ebd4db79cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:44.591819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9e2bb-31d5-4fb1-94d6-e9828325cb7f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:44.592010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9e2bb-31d5-4fb1-94d6-e9828325cb7f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:44.592254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9e2bb-31d5-4fb1-94d6-e9828325cb7f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:44.593590 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 696/1402] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:44.598315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:44.598594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:44.598861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:44.642959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6ae50aa5-1638-472e-ae60-5ebd4db79cbe None None] MFA Rules not processed for user `92173e230c3e45c3b769221fb4b3d666`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:44.644324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6ae50aa5-1638-472e-ae60-5ebd4db79cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:44.666814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:44.668016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:44.718444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:44.737249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:44.755365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 707/1403] 10.222.0.40 () {58 vars in 1040 bytes} [Wed Jul 27 08:31:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:44.765667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae50aa5-1638-472e-ae60-5ebd4db79cbe None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:44.766079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 697/1404] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:31:44.776022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae50aa5-1638-472e-ae60-5ebd4db79cbe None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:44.810432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:44.811636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:44.882335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c9f45-bbf8-4082-adb4-9914e38be785 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:44.882868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9f45-bbf8-4082-adb4-9914e38be785 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:44.883148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9f45-bbf8-4082-adb4-9914e38be785 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:44.883460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c9f45-bbf8-4082-adb4-9914e38be785 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:44.885096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b63c9f45-bbf8-4082-adb4-9914e38be785 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:44.926634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d56d6-4edc-4381-af2b-c27963aaf1be None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:44.969976 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:44.972186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c9f45-bbf8-4082-adb4-9914e38be785 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:44.973797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c9f45-bbf8-4082-adb4-9914e38be785 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:44.981934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 708/1405] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:31:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 08:31:45.015242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:45.048260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafc575-d738-4582-8502-bab7d60099e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:45.057894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafc575-d738-4582-8502-bab7d60099e3 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:45.057894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafc575-d738-4582-8502-bab7d60099e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:45.057894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafc575-d738-4582-8502-bab7d60099e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:45.057894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0aafc575-d738-4582-8502-bab7d60099e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:45.140853 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafc575-d738-4582-8502-bab7d60099e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:45.142415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aafc575-d738-4582-8502-bab7d60099e3 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:45.160350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 698/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:47.360978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fc77c-f20d-4beb-8cc2-26eecc3f18b1 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:47.361581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fc77c-f20d-4beb-8cc2-26eecc3f18b1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:47.361581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fc77c-f20d-4beb-8cc2-26eecc3f18b1 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:47.366708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 709/1407] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:31:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:31:47.368913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15519122-3d8d-4ef2-956a-265620cb80aa None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:47.369423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15519122-3d8d-4ef2-956a-265620cb80aa None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:47.369613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15519122-3d8d-4ef2-956a-265620cb80aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:47.446915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-15519122-3d8d-4ef2-956a-265620cb80aa None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:31:47.448278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-15519122-3d8d-4ef2-956a-265620cb80aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:47.505240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:47.569790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 699/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:31:47.618458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fc77c-f20d-4beb-8cc2-26eecc3f18b1 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:31:47.627187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e19fc77c-f20d-4beb-8cc2-26eecc3f18b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:47.802209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fc77c-f20d-4beb-8cc2-26eecc3f18b1 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:31:47.883978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:47.938124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f402c9a-19f5-47bb-8891-d6757f25cbd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:31:47.938897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402c9a-19f5-47bb-8891-d6757f25cbd0 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:31:47.939248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402c9a-19f5-47bb-8891-d6757f25cbd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:31:47.939588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f402c9a-19f5-47bb-8891-d6757f25cbd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:31:47.941263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9f402c9a-19f5-47bb-8891-d6757f25cbd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:31:48.053841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f402c9a-19f5-47bb-8891-d6757f25cbd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:31:48.058677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f402c9a-19f5-47bb-8891-d6757f25cbd0 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:31:48.079450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 710/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:31:57.571977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=23 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:31:58.439774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=64 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:08.653554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5eaf5-b253-4f10-bdef-a8a50aecad9d None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:08.654117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5eaf5-b253-4f10-bdef-a8a50aecad9d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:08.654117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5eaf5-b253-4f10-bdef-a8a50aecad9d None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:08.655490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 700/1410] 10.222.0.40 () {54 vars in 918 bytes} [Wed Jul 27 08:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:08.660644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c6a07-3637-49c9-961b-af7b763303a2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:08.661062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c6a07-3637-49c9-961b-af7b763303a2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:08.661205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c6a07-3637-49c9-961b-af7b763303a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:08.744676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d31c6a07-3637-49c9-961b-af7b763303a2 None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:08.745913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d31c6a07-3637-49c9-961b-af7b763303a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:08.851766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 711/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 27 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:08.865850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75b18-8aa5-4346-ad3c-33e20a2b2c9d None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:08.866785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75b18-8aa5-4346-ad3c-33e20a2b2c9d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:08.867237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75b18-8aa5-4346-ad3c-33e20a2b2c9d None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:08.868579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 701/1412] 10.222.0.40 () {54 vars in 918 bytes} [Wed Jul 27 08:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:08.873292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f479fa-f39e-4c25-b224-16fa616840f1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:08.873579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f479fa-f39e-4c25-b224-16fa616840f1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:08.873855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f479fa-f39e-4c25-b224-16fa616840f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:08.944513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:08.947514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-83f479fa-f39e-4c25-b224-16fa616840f1 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:08.948708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-83f479fa-f39e-4c25-b224-16fa616840f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:09.064458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 712/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 27 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:11.526612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d61b-34bc-4d77-b45e-6576c62948e8 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:11.527131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d61b-34bc-4d77-b45e-6576c62948e8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:11.527327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d61b-34bc-4d77-b45e-6576c62948e8 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:11.528719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:11.533862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a6a58-55c7-4e3a-8452-2c6d1abacab5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:11.534368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a6a58-55c7-4e3a-8452-2c6d1abacab5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:11.534462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a6a58-55c7-4e3a-8452-2c6d1abacab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:11.605907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f71a6a58-55c7-4e3a-8452-2c6d1abacab5 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:11.607316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f71a6a58-55c7-4e3a-8452-2c6d1abacab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:11.663063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:11.705791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:11.749648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347d61b-34bc-4d77-b45e-6576c62948e8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:11.751152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b347d61b-34bc-4d77-b45e-6576c62948e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:11.919999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347d61b-34bc-4d77-b45e-6576c62948e8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:11.996252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:12.038180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e9270-7876-4382-b1c0-8df583e91d4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:12.039115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e9270-7876-4382-b1c0-8df583e91d4c service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:12.039388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e9270-7876-4382-b1c0-8df583e91d4c service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:12.039654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e9270-7876-4382-b1c0-8df583e91d4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:12.041336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cd9e9270-7876-4382-b1c0-8df583e91d4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:12.133295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e9270-7876-4382-b1c0-8df583e91d4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:12.134840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e9270-7876-4382-b1c0-8df583e91d4c service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:12.153713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 703/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:13.074659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee04472-d5d5-4dbc-ad89-f622ffbb12e3 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:13.075194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee04472-d5d5-4dbc-ad89-f622ffbb12e3 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:13.075194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee04472-d5d5-4dbc-ad89-f622ffbb12e3 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:13.076582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:13.081787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab814bb-599d-41cb-a23e-29d9088eccb8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:13.082305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab814bb-599d-41cb-a23e-29d9088eccb8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:13.082430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab814bb-599d-41cb-a23e-29d9088eccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:13.162738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7ab814bb-599d-41cb-a23e-29d9088eccb8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:13.164540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7ab814bb-599d-41cb-a23e-29d9088eccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:13.193230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:13.266808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:32:13.420262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee04472-d5d5-4dbc-ad89-f622ffbb12e3 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:13.421850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9ee04472-d5d5-4dbc-ad89-f622ffbb12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:13.573491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee04472-d5d5-4dbc-ad89-f622ffbb12e3 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:13.614941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:13.692179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167d2b2-8e22-47f0-bb5d-196212820dee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:13.692782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167d2b2-8e22-47f0-bb5d-196212820dee service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:13.693003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167d2b2-8e22-47f0-bb5d-196212820dee service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:13.693278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167d2b2-8e22-47f0-bb5d-196212820dee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:13.695781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c167d2b2-8e22-47f0-bb5d-196212820dee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:13.789863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167d2b2-8e22-47f0-bb5d-196212820dee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:13.792823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167d2b2-8e22-47f0-bb5d-196212820dee service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:13.816402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 715/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:14.721548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7630a0-6099-4daf-94fa-1167993562fb None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:14.722069 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7630a0-6099-4daf-94fa-1167993562fb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:14.722215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7630a0-6099-4daf-94fa-1167993562fb None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:14.723533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:14.729729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583d937-8fd4-4df4-8d6a-7e65a4ba7aa1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:14.730330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583d937-8fd4-4df4-8d6a-7e65a4ba7aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:14.730330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583d937-8fd4-4df4-8d6a-7e65a4ba7aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:14.913953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:14.935939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4583d937-8fd4-4df4-8d6a-7e65a4ba7aa1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:14.937151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4583d937-8fd4-4df4-8d6a-7e65a4ba7aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:15.036123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:15.207437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7630a0-6099-4daf-94fa-1167993562fb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:15.208913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0f7630a0-6099-4daf-94fa-1167993562fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:15.364465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7630a0-6099-4daf-94fa-1167993562fb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:15.377548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:15.497138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256938e-a6e2-42f3-b6fc-9078f9e1fd6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:15.498067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256938e-a6e2-42f3-b6fc-9078f9e1fd6f service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:15.498067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256938e-a6e2-42f3-b6fc-9078f9e1fd6f service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:15.498317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256938e-a6e2-42f3-b6fc-9078f9e1fd6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:15.499913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9256938e-a6e2-42f3-b6fc-9078f9e1fd6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:15.597805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256938e-a6e2-42f3-b6fc-9078f9e1fd6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:15.599457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256938e-a6e2-42f3-b6fc-9078f9e1fd6f service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:15.617829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 706/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:16.496980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858f6ed-ce9f-4667-b212-79f41c0d41ce None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:16.497478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858f6ed-ce9f-4667-b212-79f41c0d41ce None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:16.497478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858f6ed-ce9f-4667-b212-79f41c0d41ce None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:16.499094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:16.503996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ac9f7-ae3c-45dc-855a-232d96cd7e7a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:16.504459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ac9f7-ae3c-45dc-855a-232d96cd7e7a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:16.504563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ac9f7-ae3c-45dc-855a-232d96cd7e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:16.525677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:16.584970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b77ac9f7-ae3c-45dc-855a-232d96cd7e7a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:16.586290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b77ac9f7-ae3c-45dc-855a-232d96cd7e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:16.688026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:16.842577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858f6ed-ce9f-4667-b212-79f41c0d41ce None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:16.844099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3858f6ed-ce9f-4667-b212-79f41c0d41ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:16.961460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:16.998455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858f6ed-ce9f-4667-b212-79f41c0d41ce None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:17.117381 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a92636-e07a-4cef-ad61-6dfcc6d9c5f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:17.118032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92636-e07a-4cef-ad61-6dfcc6d9c5f5 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:17.118376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92636-e07a-4cef-ad61-6dfcc6d9c5f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:17.118651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a92636-e07a-4cef-ad61-6dfcc6d9c5f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:17.120317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b5a92636-e07a-4cef-ad61-6dfcc6d9c5f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:17.203536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:17.217286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a92636-e07a-4cef-ad61-6dfcc6d9c5f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:17.218934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a92636-e07a-4cef-ad61-6dfcc6d9c5f5 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:17.237803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 718/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:18.134055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66070afe-32b5-4174-bcf4-39553b58c048 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:18.134610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66070afe-32b5-4174-bcf4-39553b58c048 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:18.134610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66070afe-32b5-4174-bcf4-39553b58c048 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:18.135955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:18.140810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bcf1b-1268-4999-8953-50aee3822462 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:18.141200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bcf1b-1268-4999-8953-50aee3822462 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:18.141365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bcf1b-1268-4999-8953-50aee3822462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:18.227489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fa3bcf1b-1268-4999-8953-50aee3822462 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:18.228724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fa3bcf1b-1268-4999-8953-50aee3822462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:18.329252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:18.482915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66070afe-32b5-4174-bcf4-39553b58c048 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:18.484430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-66070afe-32b5-4174-bcf4-39553b58c048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:18.572293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:18.636665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66070afe-32b5-4174-bcf4-39553b58c048 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:18.750687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0546ab5-02e1-43f2-a84c-316ccd7594bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:18.751228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0546ab5-02e1-43f2-a84c-316ccd7594bd service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:18.751526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0546ab5-02e1-43f2-a84c-316ccd7594bd service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:18.751783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0546ab5-02e1-43f2-a84c-316ccd7594bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:18.753398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e0546ab5-02e1-43f2-a84c-316ccd7594bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:18.805327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:18.847223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0546ab5-02e1-43f2-a84c-316ccd7594bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:18.848816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0546ab5-02e1-43f2-a84c-316ccd7594bd service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:18.868104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 709/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:19.769084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b099f-d611-41c0-a9ea-6ee2043cf5d2 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:19.769686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b099f-d611-41c0-a9ea-6ee2043cf5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:19.769686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b099f-d611-41c0-a9ea-6ee2043cf5d2 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:19.771020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:19.776485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f29db6-cedd-4c71-bc81-02f98cc9ea9a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:19.776924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f29db6-cedd-4c71-bc81-02f98cc9ea9a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:19.777020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f29db6-cedd-4c71-bc81-02f98cc9ea9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:19.852085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-37f29db6-cedd-4c71-bc81-02f98cc9ea9a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:19.853287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-37f29db6-cedd-4c71-bc81-02f98cc9ea9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:19.947009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:20.098312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b099f-d611-41c0-a9ea-6ee2043cf5d2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:20.099875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-086b099f-d611-41c0-a9ea-6ee2043cf5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:20.149295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:20.248131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b099f-d611-41c0-a9ea-6ee2043cf5d2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:20.363179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ca6a8-8a7b-417e-b9c0-12bf5a28187d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:20.363799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca6a8-8a7b-417e-b9c0-12bf5a28187d service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:20.364085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca6a8-8a7b-417e-b9c0-12bf5a28187d service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:20.364360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca6a8-8a7b-417e-b9c0-12bf5a28187d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:20.366019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-766ca6a8-8a7b-417e-b9c0-12bf5a28187d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:20.373448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:20.458043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ca6a8-8a7b-417e-b9c0-12bf5a28187d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:20.460468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ca6a8-8a7b-417e-b9c0-12bf5a28187d service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:20.478792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 721/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:21.364360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c215d-52ed-4c0c-a2c3-d9602ee0851d None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:21.364927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c215d-52ed-4c0c-a2c3-d9602ee0851d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:21.364927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c215d-52ed-4c0c-a2c3-d9602ee0851d None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:21.366895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:21.371793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c524e-d3b3-4bb3-b47e-d10497a02525 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:21.372222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c524e-d3b3-4bb3-b47e-d10497a02525 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:21.372348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c524e-d3b3-4bb3-b47e-d10497a02525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:21.447436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4f3c524e-d3b3-4bb3-b47e-d10497a02525 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:21.448635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4f3c524e-d3b3-4bb3-b47e-d10497a02525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:21.513437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:21.542012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:21.696743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c215d-52ed-4c0c-a2c3-d9602ee0851d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:21.698301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-465c215d-52ed-4c0c-a2c3-d9602ee0851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:21.716305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:21.845385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c215d-52ed-4c0c-a2c3-d9602ee0851d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:21.933323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:21.958065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0c8f3-5c26-402d-837f-2210eaa85329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:21.958734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0c8f3-5c26-402d-837f-2210eaa85329 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:21.959504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0c8f3-5c26-402d-837f-2210eaa85329 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:21.959852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0c8f3-5c26-402d-837f-2210eaa85329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:21.961594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-05f0c8f3-5c26-402d-837f-2210eaa85329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:22.049836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0c8f3-5c26-402d-837f-2210eaa85329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:22.051452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0c8f3-5c26-402d-837f-2210eaa85329 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:22.069044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 712/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:22.923606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a5594-4ea6-4cb0-90b7-585ca0b1d86a None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:22.924172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a5594-4ea6-4cb0-90b7-585ca0b1d86a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:22.924172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a5594-4ea6-4cb0-90b7-585ca0b1d86a None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:22.925496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:22.930498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57387efc-50ca-482c-9bab-e79f64a13727 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:22.930987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57387efc-50ca-482c-9bab-e79f64a13727 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:22.931096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57387efc-50ca-482c-9bab-e79f64a13727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:23.004726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-57387efc-50ca-482c-9bab-e79f64a13727 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:23.005975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-57387efc-50ca-482c-9bab-e79f64a13727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:23.047601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:23.103437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:23.253807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a5594-4ea6-4cb0-90b7-585ca0b1d86a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:23.255252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-625a5594-4ea6-4cb0-90b7-585ca0b1d86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:23.399177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a5594-4ea6-4cb0-90b7-585ca0b1d86a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:23.451726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:23.511064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ea05c-deee-400e-b9b7-502a8a1c39f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:23.511619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ea05c-deee-400e-b9b7-502a8a1c39f2 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:23.511893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ea05c-deee-400e-b9b7-502a8a1c39f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:23.512156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ea05c-deee-400e-b9b7-502a8a1c39f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:23.513744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-046ea05c-deee-400e-b9b7-502a8a1c39f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:23.601044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ea05c-deee-400e-b9b7-502a8a1c39f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:23.602679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ea05c-deee-400e-b9b7-502a8a1c39f2 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:23.620699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 724/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:24.501934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e552d-5ca7-490a-affa-1704b9ab4d3e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:24.502512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e552d-5ca7-490a-affa-1704b9ab4d3e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:24.502512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e552d-5ca7-490a-affa-1704b9ab4d3e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:24.503883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:24.509527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bfeca-21fb-4b60-a875-3b50800230c8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:24.509974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bfeca-21fb-4b60-a875-3b50800230c8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:24.510090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bfeca-21fb-4b60-a875-3b50800230c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:24.584678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-487bfeca-21fb-4b60-a875-3b50800230c8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:24.585878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-487bfeca-21fb-4b60-a875-3b50800230c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:24.587904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:24.682946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:24.833318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e552d-5ca7-490a-affa-1704b9ab4d3e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:24.835174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f00e552d-5ca7-490a-affa-1704b9ab4d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:24.980586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e552d-5ca7-490a-affa-1704b9ab4d3e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:25.006618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:25.093498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0009a5-790a-491b-8441-c3d80ad375b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:25.094041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0009a5-790a-491b-8441-c3d80ad375b6 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:25.094419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0009a5-790a-491b-8441-c3d80ad375b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:25.094692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0009a5-790a-491b-8441-c3d80ad375b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:25.096318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2c0009a5-790a-491b-8441-c3d80ad375b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:25.183736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0009a5-790a-491b-8441-c3d80ad375b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:25.185331 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0009a5-790a-491b-8441-c3d80ad375b6 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:25.203304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 715/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:27.111234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c4120-1e56-4dc2-988d-b5312262f74d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:27.111769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c4120-1e56-4dc2-988d-b5312262f74d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:27.111769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c4120-1e56-4dc2-988d-b5312262f74d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:27.113143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:27.118775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db32eb-1517-4334-8220-ebde8fef8129 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:27.119250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db32eb-1517-4334-8220-ebde8fef8129 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:27.119364 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db32eb-1517-4334-8220-ebde8fef8129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:27.160235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:27.185769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-95db32eb-1517-4334-8220-ebde8fef8129 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:27.187054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-95db32eb-1517-4334-8220-ebde8fef8129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:27.282159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:27.320768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c4120-1e56-4dc2-988d-b5312262f74d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:27.322371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7b1c4120-1e56-4dc2-988d-b5312262f74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:27.423850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c4120-1e56-4dc2-988d-b5312262f74d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:27.456497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:27.498902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375912d-5c95-4af1-bbb8-1a914adf598c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:27.499520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375912d-5c95-4af1-bbb8-1a914adf598c service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:27.499806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375912d-5c95-4af1-bbb8-1a914adf598c service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:27.500077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375912d-5c95-4af1-bbb8-1a914adf598c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:27.501689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1375912d-5c95-4af1-bbb8-1a914adf598c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:27.612797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375912d-5c95-4af1-bbb8-1a914adf598c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:27.614265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375912d-5c95-4af1-bbb8-1a914adf598c service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:27.633130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 727/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:29.788796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36743e6c-31dc-438e-850f-b5473b25ed0a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:29.789365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36743e6c-31dc-438e-850f-b5473b25ed0a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:29.789365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36743e6c-31dc-438e-850f-b5473b25ed0a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:29.790727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:29.796113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e19f34-3653-40cc-9b3d-c0f1ef3a7ff8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:29.796535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e19f34-3653-40cc-9b3d-c0f1ef3a7ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:29.796661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e19f34-3653-40cc-9b3d-c0f1ef3a7ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:29.861825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-96e19f34-3653-40cc-9b3d-c0f1ef3a7ff8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:29.863102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-96e19f34-3653-40cc-9b3d-c0f1ef3a7ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:29.875589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:29.959714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:29.998296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36743e6c-31dc-438e-850f-b5473b25ed0a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:29.999827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-36743e6c-31dc-438e-850f-b5473b25ed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:30.102960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36743e6c-31dc-438e-850f-b5473b25ed0a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:30.114700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:30.176523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109c3d1-9908-4048-8e9d-80aef68a0638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:30.177137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109c3d1-9908-4048-8e9d-80aef68a0638 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:30.177420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109c3d1-9908-4048-8e9d-80aef68a0638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:30.177696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109c3d1-9908-4048-8e9d-80aef68a0638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:30.179376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2109c3d1-9908-4048-8e9d-80aef68a0638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:30.287321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109c3d1-9908-4048-8e9d-80aef68a0638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:30.288749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109c3d1-9908-4048-8e9d-80aef68a0638 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:30.307029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 718/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:32.916436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0853b3-9a0f-49d8-a0ae-f7271d3e8835 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:32.917007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0853b3-9a0f-49d8-a0ae-f7271d3e8835 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:32.917007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0853b3-9a0f-49d8-a0ae-f7271d3e8835 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:32.918364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:32.923866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ceb86-76e4-4f8e-94fa-e7edf27fdcdc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:32.924276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ceb86-76e4-4f8e-94fa-e7edf27fdcdc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:32.924425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ceb86-76e4-4f8e-94fa-e7edf27fdcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:32.972665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:32.990951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ef8ceb86-76e4-4f8e-94fa-e7edf27fdcdc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:32.992207 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ef8ceb86-76e4-4f8e-94fa-e7edf27fdcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:33.087856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:33.128894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0853b3-9a0f-49d8-a0ae-f7271d3e8835 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:33.130543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-af0853b3-9a0f-49d8-a0ae-f7271d3e8835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:33.232232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0853b3-9a0f-49d8-a0ae-f7271d3e8835 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:33.270451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:33.304131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941716ea-2f6a-471e-b919-9baaaea27f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:33.304744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941716ea-2f6a-471e-b919-9baaaea27f70 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:33.305020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941716ea-2f6a-471e-b919-9baaaea27f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:33.305289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941716ea-2f6a-471e-b919-9baaaea27f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:33.306936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-941716ea-2f6a-471e-b919-9baaaea27f70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:33.414468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941716ea-2f6a-471e-b919-9baaaea27f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:33.415902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941716ea-2f6a-471e-b919-9baaaea27f70 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:33.433690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 730/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:35.609438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58e67f-6056-4bc3-88ac-309849c0ccc4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:35.609927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58e67f-6056-4bc3-88ac-309849c0ccc4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:35.610023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58e67f-6056-4bc3-88ac-309849c0ccc4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:35.611388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:35.617319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84a92a-9592-4f07-88fd-6dce1fe3f458 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:35.618218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84a92a-9592-4f07-88fd-6dce1fe3f458 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:35.618218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84a92a-9592-4f07-88fd-6dce1fe3f458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:35.682839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9b84a92a-9592-4f07-88fd-6dce1fe3f458 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:35.684037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9b84a92a-9592-4f07-88fd-6dce1fe3f458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:35.704398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:35.779656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:35.819303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58e67f-6056-4bc3-88ac-309849c0ccc4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:35.820892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fb58e67f-6056-4bc3-88ac-309849c0ccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:35.922558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58e67f-6056-4bc3-88ac-309849c0ccc4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:35.942504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:35.996305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf17d08-49a0-43ae-9005-0675b5ab7635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:35.996920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf17d08-49a0-43ae-9005-0675b5ab7635 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:35.997202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf17d08-49a0-43ae-9005-0675b5ab7635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:35.997476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf17d08-49a0-43ae-9005-0675b5ab7635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:35.999132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-abf17d08-49a0-43ae-9005-0675b5ab7635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:36.107207 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf17d08-49a0-43ae-9005-0675b5ab7635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:36.108647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf17d08-49a0-43ae-9005-0675b5ab7635 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:36.126841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 721/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:38.213410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a754e-36d3-4122-aa2c-9397bfc18a51 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:38.213981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a754e-36d3-4122-aa2c-9397bfc18a51 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:38.213981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a754e-36d3-4122-aa2c-9397bfc18a51 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:38.215653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:38.220939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aac04-925e-4772-bcc6-71a7e327dd45 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:38.221362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aac04-925e-4772-bcc6-71a7e327dd45 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:38.221502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aac04-925e-4772-bcc6-71a7e327dd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:38.284441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:38.287190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c74aac04-925e-4772-bcc6-71a7e327dd45 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:38.288437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c74aac04-925e-4772-bcc6-71a7e327dd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:38.384058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:38.421774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a754e-36d3-4122-aa2c-9397bfc18a51 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:38.423349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-394a754e-36d3-4122-aa2c-9397bfc18a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:38.525446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a754e-36d3-4122-aa2c-9397bfc18a51 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:38.573724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:38.598495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d619ca-f06b-4570-85ff-88a4dba4fe36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:38.599157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d619ca-f06b-4570-85ff-88a4dba4fe36 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:38.599437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d619ca-f06b-4570-85ff-88a4dba4fe36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:38.599710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d619ca-f06b-4570-85ff-88a4dba4fe36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:38.601320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-12d619ca-f06b-4570-85ff-88a4dba4fe36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:38.707685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d619ca-f06b-4570-85ff-88a4dba4fe36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:38.709202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d619ca-f06b-4570-85ff-88a4dba4fe36 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:38.728275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 733/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:40.762562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2e203-036b-48d0-9a03-c4b44aa57d70 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:40.763126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2e203-036b-48d0-9a03-c4b44aa57d70 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:40.763126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2e203-036b-48d0-9a03-c4b44aa57d70 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:40.764466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:40.769757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4287744-7ead-49bc-ab56-53fe6a040d80 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:40.770230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4287744-7ead-49bc-ab56-53fe6a040d80 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:40.770823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4287744-7ead-49bc-ab56-53fe6a040d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:40.836211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a4287744-7ead-49bc-ab56-53fe6a040d80 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:40.837423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a4287744-7ead-49bc-ab56-53fe6a040d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:40.866894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:40.942102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:40.981222 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2e203-036b-48d0-9a03-c4b44aa57d70 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:40.982756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f6d2e203-036b-48d0-9a03-c4b44aa57d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:41.085573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2e203-036b-48d0-9a03-c4b44aa57d70 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:41.113381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:41.159388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9abb73-ad32-49ed-bbe5-d153a3eb3d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:41.159997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9abb73-ad32-49ed-bbe5-d153a3eb3d5d service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:41.160274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9abb73-ad32-49ed-bbe5-d153a3eb3d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:41.160545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9abb73-ad32-49ed-bbe5-d153a3eb3d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:41.162158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ce9abb73-ad32-49ed-bbe5-d153a3eb3d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:41.268561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9abb73-ad32-49ed-bbe5-d153a3eb3d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:41.269989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9abb73-ad32-49ed-bbe5-d153a3eb3d5d service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:41.289094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 724/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:43.338793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec24a96-79f2-4fa2-9b5c-7e4b3cd569cb None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:43.339601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec24a96-79f2-4fa2-9b5c-7e4b3cd569cb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:43.339601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec24a96-79f2-4fa2-9b5c-7e4b3cd569cb None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:43.340960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:43.346089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611706d-5b26-4fec-b895-73f3453782c5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:43.346541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611706d-5b26-4fec-b895-73f3453782c5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:43.346663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611706d-5b26-4fec-b895-73f3453782c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:43.547480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d611706d-5b26-4fec-b895-73f3453782c5 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:43.548726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d611706d-5b26-4fec-b895-73f3453782c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:43.557228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:43.647783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:43.684867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec24a96-79f2-4fa2-9b5c-7e4b3cd569cb None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:43.686465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4ec24a96-79f2-4fa2-9b5c-7e4b3cd569cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:43.787832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec24a96-79f2-4fa2-9b5c-7e4b3cd569cb None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:43.842903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:43.860840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9d350-9de8-4ea4-bd32-81c8cc2924a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:43.861478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d350-9de8-4ea4-bd32-81c8cc2924a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:43.861754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d350-9de8-4ea4-bd32-81c8cc2924a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:43.862020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9d350-9de8-4ea4-bd32-81c8cc2924a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:43.863694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a0a9d350-9de8-4ea4-bd32-81c8cc2924a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:43.970221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9d350-9de8-4ea4-bd32-81c8cc2924a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:43.971613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9d350-9de8-4ea4-bd32-81c8cc2924a0 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:43.990070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 736/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:46.033184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9380-a1f1-433b-b088-86678687b784 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:46.033706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9380-a1f1-433b-b088-86678687b784 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:46.033706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9380-a1f1-433b-b088-86678687b784 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:46.035107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:46.040843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fc6f6-2353-4643-9dd3-c9274babd7f2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:46.041248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fc6f6-2353-4643-9dd3-c9274babd7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:46.041408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fc6f6-2353-4643-9dd3-c9274babd7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:46.107568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-7e7fc6f6-2353-4643-9dd3-c9274babd7f2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:46.108761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7e7fc6f6-2353-4643-9dd3-c9274babd7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:46.145766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:46.204309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:46.243396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc9380-a1f1-433b-b088-86678687b784 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:46.244916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-35dc9380-a1f1-433b-b088-86678687b784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:46.346792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc9380-a1f1-433b-b088-86678687b784 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:46.383991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:46.419414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a707ec-a54c-4f8e-90fe-8da743294644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:46.420025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a707ec-a54c-4f8e-90fe-8da743294644 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:46.420307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a707ec-a54c-4f8e-90fe-8da743294644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:46.420576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a707ec-a54c-4f8e-90fe-8da743294644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:46.422917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54a707ec-a54c-4f8e-90fe-8da743294644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:46.530250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a707ec-a54c-4f8e-90fe-8da743294644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:46.531592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a707ec-a54c-4f8e-90fe-8da743294644 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:46.549559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 727/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:48.550447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb24fe-c518-4899-9d62-c1345bdc1d0e None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:48.551018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb24fe-c518-4899-9d62-c1345bdc1d0e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:48.551018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb24fe-c518-4899-9d62-c1345bdc1d0e None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:48.552960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:48.558444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385629-eb41-4314-960e-c6fed21b92f3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:48.558874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385629-eb41-4314-960e-c6fed21b92f3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:48.558978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6385629-eb41-4314-960e-c6fed21b92f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:48.625585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f6385629-eb41-4314-960e-c6fed21b92f3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:48.626870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f6385629-eb41-4314-960e-c6fed21b92f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:48.644649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:48.721981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:48.759259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb24fe-c518-4899-9d62-c1345bdc1d0e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:48.760772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ecfb24fe-c518-4899-9d62-c1345bdc1d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:48.861940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb24fe-c518-4899-9d62-c1345bdc1d0e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:48.927092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:48.933552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5cad86-979d-41da-a94f-d13ac7457a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:48.934185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cad86-979d-41da-a94f-d13ac7457a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:48.941536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cad86-979d-41da-a94f-d13ac7457a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:48.941536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5cad86-979d-41da-a94f-d13ac7457a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:48.941536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6b5cad86-979d-41da-a94f-d13ac7457a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:49.045144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cad86-979d-41da-a94f-d13ac7457a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:49.046623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5cad86-979d-41da-a94f-d13ac7457a74 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:49.064447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 739/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:32:51.081365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca1a29-8f93-4ab1-819b-f3e278ec8792 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:51.081883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca1a29-8f93-4ab1-819b-f3e278ec8792 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:51.081883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca1a29-8f93-4ab1-819b-f3e278ec8792 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:51.083897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:32:51.089311 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b8064-7e60-43e3-85cd-bdca410c6313 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:51.089760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b8064-7e60-43e3-85cd-bdca410c6313 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:51.089882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b8064-7e60-43e3-85cd-bdca410c6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:51.156202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-8a6b8064-7e60-43e3-85cd-bdca410c6313 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:32:51.159084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8a6b8064-7e60-43e3-85cd-bdca410c6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:51.206725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:51.259635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:32:51.296690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca1a29-8f93-4ab1-819b-f3e278ec8792 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:32:51.298289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9bca1a29-8f93-4ab1-819b-f3e278ec8792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:51.403129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca1a29-8f93-4ab1-819b-f3e278ec8792 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:32:51.450501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:32:51.477126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f05e-20d5-4673-9f76-86c77c428a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:32:51.477738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f05e-20d5-4673-9f76-86c77c428a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:32:51.478013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f05e-20d5-4673-9f76-86c77c428a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:32:51.478313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f05e-20d5-4673-9f76-86c77c428a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:32:51.479936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4de9f05e-20d5-4673-9f76-86c77c428a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:32:51.588205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f05e-20d5-4673-9f76-86c77c428a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:32:51.589892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f05e-20d5-4673-9f76-86c77c428a46 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:32:51.608753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 730/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:33:01.257768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=22 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:01.606480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=66 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:11.382354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028da06e-9025-4e9c-a765-a67566056b0d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:11.382867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028da06e-9025-4e9c-a765-a67566056b0d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:11.382867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028da06e-9025-4e9c-a765-a67566056b0d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:11.384229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 741/1471] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:11.389555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e14ac-54d7-42af-901f-308b387931b9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:11.389977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e14ac-54d7-42af-901f-308b387931b9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:11.390082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e14ac-54d7-42af-901f-308b387931b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:11.454208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5d3e14ac-54d7-42af-901f-308b387931b9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:11.455397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d3e14ac-54d7-42af-901f-308b387931b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:11.550494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 731/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:11.634797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0e370-9386-4c5b-b2d2-ce591684a321 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:11.635025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0e370-9386-4c5b-b2d2-ce591684a321 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:11.635293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0e370-9386-4c5b-b2d2-ce591684a321 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:11.636661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 742/1473] 10.222.0.40 () {54 vars in 966 bytes} [Wed Jul 27 08:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:11.641404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fa723-89a7-4aba-8e0e-d513b284eec1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:11.641689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fa723-89a7-4aba-8e0e-d513b284eec1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:11.642826 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fa723-89a7-4aba-8e0e-d513b284eec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:11.716998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-280fa723-89a7-4aba-8e0e-d513b284eec1 None None] MFA Rules not processed for user `4e41212825ad4990958a7e909c669177`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:11.718229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-280fa723-89a7-4aba-8e0e-d513b284eec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:11.739277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:11.794166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 732/1474] 10.222.0.40 () {58 vars in 1063 bytes} [Wed Jul 27 08:33:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:11.802213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0e370-9386-4c5b-b2d2-ce591684a321 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:33:11.803765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b3a0e370-9386-4c5b-b2d2-ce591684a321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:11.907575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0e370-9386-4c5b-b2d2-ce591684a321 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:33:11.979389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6264b70-e183-4951-8ae5-6bc514992a82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:33:11.979992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6264b70-e183-4951-8ae5-6bc514992a82 service glance] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:11.980275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6264b70-e183-4951-8ae5-6bc514992a82 service glance] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:11.980545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6264b70-e183-4951-8ae5-6bc514992a82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:11.982169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b6264b70-e183-4951-8ae5-6bc514992a82 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:12.036209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:12.090539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6264b70-e183-4951-8ae5-6bc514992a82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:33:12.091965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6264b70-e183-4951-8ae5-6bc514992a82 service glance] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:33:12.110000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 743/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:33:14.204139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e5b0-b518-46b5-85d7-ee9d6f9eb83b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:14.204687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e5b0-b518-46b5-85d7-ee9d6f9eb83b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:14.204687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e5b0-b518-46b5-85d7-ee9d6f9eb83b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:14.206680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 733/1476] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:14.211986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dc92-242b-4933-a89b-e8ad463acaf8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:14.212377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dc92-242b-4933-a89b-e8ad463acaf8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:14.212526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38dc92-242b-4933-a89b-e8ad463acaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:14.276290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ec38dc92-242b-4933-a89b-e8ad463acaf8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:14.277471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ec38dc92-242b-4933-a89b-e8ad463acaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:14.372935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 744/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:14.396620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e5b0-b518-46b5-85d7-ee9d6f9eb83b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:33:14.398131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bb20e5b0-b518-46b5-85d7-ee9d6f9eb83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:14.520581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e5b0-b518-46b5-85d7-ee9d6f9eb83b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:33:14.568904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:14.613620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a0780-7761-4111-9b3a-d0943854c766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:33:14.614255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a0780-7761-4111-9b3a-d0943854c766 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:14.614551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a0780-7761-4111-9b3a-d0943854c766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:14.614797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a0780-7761-4111-9b3a-d0943854c766 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:14.615505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a0780-7761-4111-9b3a-d0943854c766 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:33:14.617198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-ed9a0780-7761-4111-9b3a-d0943854c766 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:33:14.617198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:33:14.617198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:33:14.618356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a0780-7761-4111-9b3a-d0943854c766 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:33:14.637890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 734/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 27 08:33:14 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:33:16.674751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd171879-6f83-4b80-9fc8-6df88e64c070 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:16.675330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd171879-6f83-4b80-9fc8-6df88e64c070 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:16.675330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd171879-6f83-4b80-9fc8-6df88e64c070 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:16.676652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 745/1479] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:16.681815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7fef3-5a32-4ba6-b4f4-1efadec1c77f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:16.682299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7fef3-5a32-4ba6-b4f4-1efadec1c77f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:16.682407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7fef3-5a32-4ba6-b4f4-1efadec1c77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:16.748638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-aeb7fef3-5a32-4ba6-b4f4-1efadec1c77f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:16.749842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-aeb7fef3-5a32-4ba6-b4f4-1efadec1c77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:16.844882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 735/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:16.888675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd171879-6f83-4b80-9fc8-6df88e64c070 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:33:16.890213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bd171879-6f83-4b80-9fc8-6df88e64c070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:16.900030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:17.034107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd171879-6f83-4b80-9fc8-6df88e64c070 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:33:17.113845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:17.145218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0eb23-ba3d-420d-b87a-d2c7a619e03e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:33:17.145827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0eb23-ba3d-420d-b87a-d2c7a619e03e service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:17.146105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0eb23-ba3d-420d-b87a-d2c7a619e03e service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:17.146403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0eb23-ba3d-420d-b87a-d2c7a619e03e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:17.148006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5ea0eb23-ba3d-420d-b87a-d2c7a619e03e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:17.237861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0eb23-ba3d-420d-b87a-d2c7a619e03e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:33:17.239498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0eb23-ba3d-420d-b87a-d2c7a619e03e service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:33:17.257545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 746/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:33:19.437563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ba9-0509-48a6-8fc3-93f8bf6361c2 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:19.438090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ba9-0509-48a6-8fc3-93f8bf6361c2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:19.438090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ba9-0509-48a6-8fc3-93f8bf6361c2 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:19.439407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 736/1482] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:19.444577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbcc7d-7d46-4154-8a9d-816c1a8f40bb None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:19.444971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbcc7d-7d46-4154-8a9d-816c1a8f40bb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:19.445138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbcc7d-7d46-4154-8a9d-816c1a8f40bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:19.510316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-49cbcc7d-7d46-4154-8a9d-816c1a8f40bb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:19.511532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-49cbcc7d-7d46-4154-8a9d-816c1a8f40bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:19.558758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:19.608276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 747/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:19.644597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ba9-0509-48a6-8fc3-93f8bf6361c2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:33:19.646165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-50b15ba9-0509-48a6-8fc3-93f8bf6361c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:19.670357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:19.750873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ba9-0509-48a6-8fc3-93f8bf6361c2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:33:19.822771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a6391-3106-49a0-90eb-c35d40dee0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:33:19.823385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a6391-3106-49a0-90eb-c35d40dee0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:19.823665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a6391-3106-49a0-90eb-c35d40dee0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:19.823937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a6391-3106-49a0-90eb-c35d40dee0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:19.825548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fd5a6391-3106-49a0-90eb-c35d40dee0b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:19.900070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:19.935860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a6391-3106-49a0-90eb-c35d40dee0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:33:19.937362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a6391-3106-49a0-90eb-c35d40dee0b6 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:33:19.956003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 737/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:33:22.006987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884fc19-f5d6-45df-99f7-f50b1e0db519 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:22.007597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884fc19-f5d6-45df-99f7-f50b1e0db519 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:22.007597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884fc19-f5d6-45df-99f7-f50b1e0db519 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:22.008903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 748/1485] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:22.013993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94775b-4786-40ac-86b1-0051fbe564a0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:22.014466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94775b-4786-40ac-86b1-0051fbe564a0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:22.014594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94775b-4786-40ac-86b1-0051fbe564a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:22.082786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bf94775b-4786-40ac-86b1-0051fbe564a0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:22.084003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bf94775b-4786-40ac-86b1-0051fbe564a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:22.184986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 738/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:22.224747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884fc19-f5d6-45df-99f7-f50b1e0db519 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:33:22.226347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5884fc19-f5d6-45df-99f7-f50b1e0db519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:22.331453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884fc19-f5d6-45df-99f7-f50b1e0db519 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:33:22.401244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:22.403026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7276f-9a68-4714-99df-096cc4741d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:33:22.403670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7276f-9a68-4714-99df-096cc4741d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:22.403945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7276f-9a68-4714-99df-096cc4741d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:22.404220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7276f-9a68-4714-99df-096cc4741d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:22.405816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4ca7276f-9a68-4714-99df-096cc4741d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:22.512543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7276f-9a68-4714-99df-096cc4741d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:33:22.514007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7276f-9a68-4714-99df-096cc4741d2e service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:33:22.531543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 749/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:33:24.687045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcb67e-fde2-4230-9c1c-faa27e3052c1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:24.687619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcb67e-fde2-4230-9c1c-faa27e3052c1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:24.687619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcb67e-fde2-4230-9c1c-faa27e3052c1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:24.688994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 739/1488] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:24.694335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa1a71-efe7-4bf2-b23f-7b69e902823a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:24.694770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa1a71-efe7-4bf2-b23f-7b69e902823a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:24.694888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa1a71-efe7-4bf2-b23f-7b69e902823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:24.759474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-adaa1a71-efe7-4bf2-b23f-7b69e902823a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:24.760657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-adaa1a71-efe7-4bf2-b23f-7b69e902823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:24.812121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:24.856449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 750/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:24.896700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcb67e-fde2-4230-9c1c-faa27e3052c1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:33:24.898287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7ddcb67e-fde2-4230-9c1c-faa27e3052c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:24.934156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:25.001155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcb67e-fde2-4230-9c1c-faa27e3052c1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:33:25.074502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c727fb5-d4fe-4404-8469-70bdb4966365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:33:25.075109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727fb5-d4fe-4404-8469-70bdb4966365 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:25.075387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727fb5-d4fe-4404-8469-70bdb4966365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:25.075652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727fb5-d4fe-4404-8469-70bdb4966365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:25.077251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1c727fb5-d4fe-4404-8469-70bdb4966365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:25.156574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:25.185199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c727fb5-d4fe-4404-8469-70bdb4966365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:33:25.186731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c727fb5-d4fe-4404-8469-70bdb4966365 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:33:25.205551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 740/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:33:27.581016 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04b31b-e5b0-4597-bda7-e820eba6856c None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:27.581489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04b31b-e5b0-4597-bda7-e820eba6856c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:27.581613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04b31b-e5b0-4597-bda7-e820eba6856c None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:27.582961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 751/1491] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 27 08:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:33:27.588277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abf3d2-913a-4eb6-a977-a5ea84fc58ff None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:27.590659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abf3d2-913a-4eb6-a977-a5ea84fc58ff None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:27.590659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abf3d2-913a-4eb6-a977-a5ea84fc58ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:27.655120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-84abf3d2-913a-4eb6-a977-a5ea84fc58ff None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:33:27.656369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-84abf3d2-913a-4eb6-a977-a5ea84fc58ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:27.753199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 741/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 27 08:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:33:27.791736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04b31b-e5b0-4597-bda7-e820eba6856c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:33:27.793233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ba04b31b-e5b0-4597-bda7-e820eba6856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:27.894742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04b31b-e5b0-4597-bda7-e820eba6856c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:33:27.967079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e9bbf-19c3-4ea1-9cd4-dd389a871fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:33:27.967682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e9bbf-19c3-4ea1-9cd4-dd389a871fef service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:33:27.967953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e9bbf-19c3-4ea1-9cd4-dd389a871fef service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:33:27.968217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e9bbf-19c3-4ea1-9cd4-dd389a871fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:33:27.969800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3b0e9bbf-19c3-4ea1-9cd4-dd389a871fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:33:27.979292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:28.077859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e9bbf-19c3-4ea1-9cd4-dd389a871fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:33:28.079328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e9bbf-19c3-4ea1-9cd4-dd389a871fef service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:33:28.097970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 752/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:33:37.752000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=82 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:33:38.095708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=50 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:35:29.004470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf1d3-c102-4626-8190-966bd1a22448 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.005085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf1d3-c102-4626-8190-966bd1a22448 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.005085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf1d3-c102-4626-8190-966bd1a22448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.084906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5cedf1d3-c102-4626-8190-966bd1a22448 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:35:29.086099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5cedf1d3-c102-4626-8190-966bd1a22448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:35:29.192992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 742/1494] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:35:29.201957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16421-e3d4-413c-9bd8-d8302a265680 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.202510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16421-e3d4-413c-9bd8-d8302a265680 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.202510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee16421-e3d4-413c-9bd8-d8302a265680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.222220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fee16421-e3d4-413c-9bd8-d8302a265680 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:35:29.223413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fee16421-e3d4-413c-9bd8-d8302a265680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:35:29.231996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 753/1495] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:35:29.236464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf1d3-c102-4626-8190-966bd1a22448 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:29.246589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf1d3-c102-4626-8190-966bd1a22448 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:29.251349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:29.251950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.252228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.252502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.253191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:35:29.253496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:29.254644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:29.254644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:35:29.254644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:35:29.255472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:29.267692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 743/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:35:29.275657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16421-e3d4-413c-9bd8-d8302a265680 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:29.284974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee16421-e3d4-413c-9bd8-d8302a265680 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:29.289488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:29.290092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.290397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.290666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.291315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:29.294071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING py.warnings [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:29.294071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: warnings.warn(msg) Jul 27 08:35:29.294071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: Jul 27 08:35:29.295286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:29.441712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 754/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:35:29 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:35:29.446052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:29.456046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439ec52-c8d2-4a93-9543-3be9222d0727 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:29.505702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:35:29.512466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:29.517305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.517305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.517305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.517305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:29.517305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:29.517305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:35:29.517305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:35:29.517870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:29.594213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 744/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:35:29 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:35:29.598587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:29.609458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842ab48-74ad-42a2-a989-d216fbff2d00 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:29.614065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:29.614702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.614982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.618670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.618670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:29.618670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING py.warnings [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:29.618670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: warnings.warn(msg) Jul 27 08:35:29.618670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: Jul 27 08:35:29.618670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:29.631541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 755/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:35:29.635601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:29.646093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953f2ce-0f24-4a70-8f63-9b31bbc53ae8 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:29.650779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8d67b-a05a-4861-8c6f-00a4e9df5358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:29.651384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8d67b-a05a-4861-8c6f-00a4e9df5358 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.651664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8d67b-a05a-4861-8c6f-00a4e9df5358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.651948 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8d67b-a05a-4861-8c6f-00a4e9df5358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.652665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8d67b-a05a-4861-8c6f-00a4e9df5358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:29.653658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-8fc8d67b-a05a-4861-8c6f-00a4e9df5358 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:29.653658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:35:29.653658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:35:29.654573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8d67b-a05a-4861-8c6f-00a4e9df5358 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:29.668437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 745/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:35:29.672699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:29.682491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278d8b8-ff57-43d9-9187-62dbac9a9022 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:29.687652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae2c5a-f3a1-483b-a809-d2b776d8c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:29.688354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae2c5a-f3a1-483b-a809-d2b776d8c872 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.688638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae2c5a-f3a1-483b-a809-d2b776d8c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.688911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae2c5a-f3a1-483b-a809-d2b776d8c872 admin admin] PATH_INFO: `/v3/projects/639ad36d8d3547ae93adf71dc97ac574/users/76d7d08fd2aa44638a3458c0a843bbb1/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.703444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae2c5a-f3a1-483b-a809-d2b776d8c872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=639ad36d8d3547ae93adf71dc97ac574, user_id=76d7d08fd2aa44638a3458c0a843bbb1, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:29.704518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING py.warnings [None req-7eae2c5a-f3a1-483b-a809-d2b776d8c872 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:29.704518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: warnings.warn(msg) Jul 27 08:35:29.704518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: Jul 27 08:35:29.706504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae2c5a-f3a1-483b-a809-d2b776d8c872 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:29.765126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 756/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:35:29 2022] PUT /identity/v3/projects/639ad36d8d3547ae93adf71dc97ac574/users/76d7d08fd2aa44638a3458c0a843bbb1/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:35:29.778245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea4cb8-b305-46d1-ae2c-9ab1fab1b6d7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.778528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea4cb8-b305-46d1-ae2c-9ab1fab1b6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.778797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea4cb8-b305-46d1-ae2c-9ab1fab1b6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.806253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9fea4cb8-b305-46d1-ae2c-9ab1fab1b6d7 None None] MFA Rules not processed for user `76d7d08fd2aa44638a3458c0a843bbb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:35:29.807443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9fea4cb8-b305-46d1-ae2c-9ab1fab1b6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:35:29.868996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 746/1502] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:35:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:35:29.882204 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988bce-6bb2-419a-b3bb-67485e01470d None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.882497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988bce-6bb2-419a-b3bb-67485e01470d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.882776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988bce-6bb2-419a-b3bb-67485e01470d None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.884121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 757/1503] 10.222.0.40 () {54 vars in 964 bytes} [Wed Jul 27 08:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:35:29.888760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ab514-328a-4c3c-bb2d-b1d92e31ee6e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:29.889045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ab514-328a-4c3c-bb2d-b1d92e31ee6e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:29.889319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ab514-328a-4c3c-bb2d-b1d92e31ee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:29.948566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5a1ab514-328a-4c3c-bb2d-b1d92e31ee6e None None] MFA Rules not processed for user `99a29a23f8a84076bab3adefc042a088`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:35:29.949858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5a1ab514-328a-4c3c-bb2d-b1d92e31ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:35:30.023343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 747/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Jul 27 08:35:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:35:30.032076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988bce-6bb2-419a-b3bb-67485e01470d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:30.042452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988bce-6bb2-419a-b3bb-67485e01470d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:30.046649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:30.047243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:30.047524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:30.047800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:30.055908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:30.057334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:30.066664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 758/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:35:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:35:30.111307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72b9b-b10b-43b8-a837-c81a7693d513 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:30.111593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72b9b-b10b-43b8-a837-c81a7693d513 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:30.111869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc72b9b-b10b-43b8-a837-c81a7693d513 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:30.113125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 748/1506] 10.222.0.40 () {52 vars in 830 bytes} [Wed Jul 27 08:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:35:30.116557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:30.178911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ee47c-6dc3-457f-89aa-6fb817a6b0c3 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:30.184012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:30.184658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:30.184932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:30.185202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] PATH_INFO: `/v3/users/76d7d08fd2aa44638a3458c0a843bbb1` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:30.186383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d7d08fd2aa44638a3458c0a843bbb1)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:30.187401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING py.warnings [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:30.187401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: warnings.warn(msg) Jul 27 08:35:30.187401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: Jul 27 08:35:30.188633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:30.416820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d7d08fd2aa44638a3458c0a843bbb1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:30.436464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d7d08fd2aa44638a3458c0a843bbb1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:30.530527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d7d08fd2aa44638a3458c0a843bbb1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:30.531230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d7d08fd2aa44638a3458c0a843bbb1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:30.531830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d7d08fd2aa44638a3458c0a843bbb1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:30.540513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:35:30.553201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 759/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:35:30 2022] DELETE /identity/v3/users/76d7d08fd2aa44638a3458c0a843bbb1 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:35:30.557430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:30.557993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:30.558465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:30.577224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:35:30.578444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:35:30.587654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:35:30.637874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 749/1508] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:35:30.647515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:30.648894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:35:30.726422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f46853-8db7-4a90-96c2-f715e939f184 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:30.729893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1abfa-88a6-46de-82d7-ca5c673f0022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:30.731187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1abfa-88a6-46de-82d7-ca5c673f0022 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:30.731462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1abfa-88a6-46de-82d7-ca5c673f0022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:30.731731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1abfa-88a6-46de-82d7-ca5c673f0022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:30.740556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1abfa-88a6-46de-82d7-ca5c673f0022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:30.741988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1abfa-88a6-46de-82d7-ca5c673f0022 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:30.751406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 760/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:35:31.201706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:35:31.203309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:35:31.216979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd7761-ecc4-4dc1-ba02-6ebc2ba636f0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:35:31.222247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:35:31.222687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:35:31.223063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:35:31.223232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] PATH_INFO: `/v3/projects/639ad36d8d3547ae93adf71dc97ac574` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:35:31.224396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=639ad36d8d3547ae93adf71dc97ac574)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:35:31.225645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:35:31.225645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:35:31.225645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:35:31.226965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:35:31.376674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639ad36d8d3547ae93adf71dc97ac574', 'request_id': 'req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:31.382872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639ad36d8d3547ae93adf71dc97ac574', 'request_id': 'req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:31.397027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639ad36d8d3547ae93adf71dc97ac574', 'request_id': 'req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:31.502312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] The token cache is being invalidate because project 639ad36d8d3547ae93adf71dc97ac574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:35:31.502517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:31.503273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-75f85908-0cf5-47f7-bfcd-45dbf7312e8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:35:31.504782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 750/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:35:31 2022] DELETE /identity/v3/projects/639ad36d8d3547ae93adf71dc97ac574 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:35:40.747322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=38 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:35:41.499178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:37:15.042075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758c03c-c37d-4dd0-8729-d8e6c53bde71 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:15.042699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758c03c-c37d-4dd0-8729-d8e6c53bde71 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:15.042699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758c03c-c37d-4dd0-8729-d8e6c53bde71 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:15.044015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:15.048877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743da444-fe25-4f61-a92f-c3f75ff9fe84 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:15.049329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743da444-fe25-4f61-a92f-c3f75ff9fe84 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:15.049479 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743da444-fe25-4f61-a92f-c3f75ff9fe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:15.068501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-743da444-fe25-4f61-a92f-c3f75ff9fe84 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:15.069732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-743da444-fe25-4f61-a92f-c3f75ff9fe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:15.129534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:15.284132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758c03c-c37d-4dd0-8729-d8e6c53bde71 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:15.285831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1758c03c-c37d-4dd0-8729-d8e6c53bde71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:15.388734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758c03c-c37d-4dd0-8729-d8e6c53bde71 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:15.394389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39138250-dd6c-4e67-8dab-d28a4224ec19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:15.395011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39138250-dd6c-4e67-8dab-d28a4224ec19 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:15.395288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39138250-dd6c-4e67-8dab-d28a4224ec19 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:15.395554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39138250-dd6c-4e67-8dab-d28a4224ec19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:15.404367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39138250-dd6c-4e67-8dab-d28a4224ec19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:15.406165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39138250-dd6c-4e67-8dab-d28a4224ec19 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:15.415927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 762/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:16.257622 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd09a9-f20e-4dfa-8f21-7fe5cd26620a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:16.258269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd09a9-f20e-4dfa-8f21-7fe5cd26620a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:16.258269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd09a9-f20e-4dfa-8f21-7fe5cd26620a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:16.260283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:16.265091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d95ae-72cf-4844-bde0-4a48d0a04633 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:16.265541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d95ae-72cf-4844-bde0-4a48d0a04633 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:16.265668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d95ae-72cf-4844-bde0-4a48d0a04633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:16.285150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e40d95ae-72cf-4844-bde0-4a48d0a04633 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:16.286374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e40d95ae-72cf-4844-bde0-4a48d0a04633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:16.294181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:16.445759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd09a9-f20e-4dfa-8f21-7fe5cd26620a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:16.461290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd09a9-f20e-4dfa-8f21-7fe5cd26620a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:16.467311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911dbcd-101d-471e-8681-477770ac2afb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:16.467921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911dbcd-101d-471e-8681-477770ac2afb service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:16.468200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911dbcd-101d-471e-8681-477770ac2afb service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:16.468472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911dbcd-101d-471e-8681-477770ac2afb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:16.476708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911dbcd-101d-471e-8681-477770ac2afb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:16.478539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911dbcd-101d-471e-8681-477770ac2afb service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:16.487786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 753/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:16.543086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca8dc-dcc1-413d-aca1-8110c0eab40c None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:16.543375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca8dc-dcc1-413d-aca1-8110c0eab40c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:16.543652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ca8dc-dcc1-413d-aca1-8110c0eab40c None None] PATH_INFO: `/v3` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:16.545077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 764/1517] 10.222.0.42 () {54 vars in 925 bytes} [Wed Jul 27 08:37:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:37:16.554961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88839a-7cdc-4342-9fed-85129e7adc86 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:16.555248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88839a-7cdc-4342-9fed-85129e7adc86 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:16.555528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88839a-7cdc-4342-9fed-85129e7adc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:16.608452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2b88839a-7cdc-4342-9fed-85129e7adc86 None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:16.609652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2b88839a-7cdc-4342-9fed-85129e7adc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:16.680343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 754/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 27 08:37:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:37:26.111976 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4e52-be30-4f00-9da9-054f32457798 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:26.112495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4e52-be30-4f00-9da9-054f32457798 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:26.112495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4e52-be30-4f00-9da9-054f32457798 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:26.113772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 765/1519] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 27 08:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:26.118645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62d33-9a47-41fc-b3a5-a08cff6e9a8f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:26.119053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62d33-9a47-41fc-b3a5-a08cff6e9a8f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:26.119204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e62d33-9a47-41fc-b3a5-a08cff6e9a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:26.139111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-98e62d33-9a47-41fc-b3a5-a08cff6e9a8f None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:26.140305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-98e62d33-9a47-41fc-b3a5-a08cff6e9a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:26.148735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 755/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 27 08:37:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:26.156229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a4e52-be30-4f00-9da9-054f32457798 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:26.169547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a4e52-be30-4f00-9da9-054f32457798 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:26.175819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cf4b3-c5e0-4404-b939-ef97e31c2d7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:26.176436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cf4b3-c5e0-4404-b939-ef97e31c2d7e service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:26.176858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cf4b3-c5e0-4404-b939-ef97e31c2d7e service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:26.177111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cf4b3-c5e0-4404-b939-ef97e31c2d7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:26.186252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cf4b3-c5e0-4404-b939-ef97e31c2d7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:26.187876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cf4b3-c5e0-4404-b939-ef97e31c2d7e service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:26.197601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 766/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:27.723928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ca6b-d3ad-402b-8d5e-0cf1a7a09a90 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:27.724499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ca6b-d3ad-402b-8d5e-0cf1a7a09a90 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:27.724499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287ca6b-d3ad-402b-8d5e-0cf1a7a09a90 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:27.725822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:27.731524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012237e-220f-4f43-8164-e55d7482fff2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:27.731952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012237e-220f-4f43-8164-e55d7482fff2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:27.732356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012237e-220f-4f43-8164-e55d7482fff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:27.751796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e012237e-220f-4f43-8164-e55d7482fff2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:27.752977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e012237e-220f-4f43-8164-e55d7482fff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:27.760759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:27.911507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287ca6b-d3ad-402b-8d5e-0cf1a7a09a90 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:27.926654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287ca6b-d3ad-402b-8d5e-0cf1a7a09a90 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:27.931911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1eb3a-48f2-4f23-b161-f911a03d5448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:27.932508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1eb3a-48f2-4f23-b161-f911a03d5448 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:27.933372 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1eb3a-48f2-4f23-b161-f911a03d5448 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:27.933642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1eb3a-48f2-4f23-b161-f911a03d5448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:27.941609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1eb3a-48f2-4f23-b161-f911a03d5448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:27.943204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1eb3a-48f2-4f23-b161-f911a03d5448 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:27.952512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 757/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:28.765709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9471-4619-4663-8bff-5bd63bce16ae None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:28.766314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9471-4619-4663-8bff-5bd63bce16ae None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:28.766314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9471-4619-4663-8bff-5bd63bce16ae None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:28.767648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 27 08:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:28.772538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383423b7-1f50-485f-9a0f-c4592de296dc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:28.772965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383423b7-1f50-485f-9a0f-c4592de296dc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:28.773079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383423b7-1f50-485f-9a0f-c4592de296dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:28.792791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-383423b7-1f50-485f-9a0f-c4592de296dc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:28.793992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-383423b7-1f50-485f-9a0f-c4592de296dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:28.801930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 27 08:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:28.955533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea9471-4619-4663-8bff-5bd63bce16ae None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:28.970324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea9471-4619-4663-8bff-5bd63bce16ae None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:28.975576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2e548-cb4a-43a2-9ed0-f95cdfe0a2b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:28.976167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2e548-cb4a-43a2-9ed0-f95cdfe0a2b2 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:28.976442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2e548-cb4a-43a2-9ed0-f95cdfe0a2b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:28.976714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2e548-cb4a-43a2-9ed0-f95cdfe0a2b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:28.985206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2e548-cb4a-43a2-9ed0-f95cdfe0a2b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:28.986815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2e548-cb4a-43a2-9ed0-f95cdfe0a2b2 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:28.996057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 769/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:33.076436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b5be1-3783-4192-95f4-2eae1089d21f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:33.076989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b5be1-3783-4192-95f4-2eae1089d21f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:33.076989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b5be1-3783-4192-95f4-2eae1089d21f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:33.078370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:33.083382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd6d52-21e0-43b8-8c8c-b5720f8e2947 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:33.083793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd6d52-21e0-43b8-8c8c-b5720f8e2947 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:33.083919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd6d52-21e0-43b8-8c8c-b5720f8e2947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:33.103126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-27bd6d52-21e0-43b8-8c8c-b5720f8e2947 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:33.104832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-27bd6d52-21e0-43b8-8c8c-b5720f8e2947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:33.112558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:33.185306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b5be1-3783-4192-95f4-2eae1089d21f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:33.187051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c85b5be1-3783-4192-95f4-2eae1089d21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:33.215115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:37:33.243687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b5be1-3783-4192-95f4-2eae1089d21f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:33.248211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9129313-9bbd-43ef-9bbd-e5401251c465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:33.248824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9129313-9bbd-43ef-9bbd-e5401251c465 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:33.249101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9129313-9bbd-43ef-9bbd-e5401251c465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:33.249419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9129313-9bbd-43ef-9bbd-e5401251c465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:33.258827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9129313-9bbd-43ef-9bbd-e5401251c465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:33.260249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9129313-9bbd-43ef-9bbd-e5401251c465 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:33.269663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 760/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:35.277670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed466-9370-4414-b5ac-04e071c9f4bb None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:35.278205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed466-9370-4414-b5ac-04e071c9f4bb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:35.278303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ed466-9370-4414-b5ac-04e071c9f4bb None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:35.279612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:35.284815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c506f-96c1-47e9-9bc5-414b4db8d189 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:35.285244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c506f-96c1-47e9-9bc5-414b4db8d189 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:35.285433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c506f-96c1-47e9-9bc5-414b4db8d189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:35.304786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d61c506f-96c1-47e9-9bc5-414b4db8d189 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:35.306002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d61c506f-96c1-47e9-9bc5-414b4db8d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:35.314361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:35.350697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ed466-9370-4414-b5ac-04e071c9f4bb None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:35.361856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ed466-9370-4414-b5ac-04e071c9f4bb None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:35.365335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1695c6b-1220-404a-ab2b-7dd500c01c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:35.365931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1695c6b-1220-404a-ab2b-7dd500c01c63 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:35.366317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1695c6b-1220-404a-ab2b-7dd500c01c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:35.366594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1695c6b-1220-404a-ab2b-7dd500c01c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:35.375950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1695c6b-1220-404a-ab2b-7dd500c01c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:35.377379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1695c6b-1220-404a-ab2b-7dd500c01c63 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:35.387451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 772/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:39.537897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18604992-3951-4238-88a6-512e8e1c4d0c None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:39.538447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18604992-3951-4238-88a6-512e8e1c4d0c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:39.538447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18604992-3951-4238-88a6-512e8e1c4d0c None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:39.539767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:39.544959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930a7c-a282-40b5-aa8c-582775dfcc88 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:39.545403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930a7c-a282-40b5-aa8c-582775dfcc88 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:39.545510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930a7c-a282-40b5-aa8c-582775dfcc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:39.565331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-96930a7c-a282-40b5-aa8c-582775dfcc88 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:39.566580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-96930a7c-a282-40b5-aa8c-582775dfcc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:39.575018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:39.881555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18604992-3951-4238-88a6-512e8e1c4d0c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:39.883289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-18604992-3951-4238-88a6-512e8e1c4d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:39.941354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18604992-3951-4238-88a6-512e8e1c4d0c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:39.945001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4fc50-24c9-4bd5-bda7-b36440138a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:39.945623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4fc50-24c9-4bd5-bda7-b36440138a72 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:39.945904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4fc50-24c9-4bd5-bda7-b36440138a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:39.946200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4fc50-24c9-4bd5-bda7-b36440138a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:39.955351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4fc50-24c9-4bd5-bda7-b36440138a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:39.956786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4fc50-24c9-4bd5-bda7-b36440138a72 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:39.966384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 763/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:42.004768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c542-f0e7-4f2f-b90f-fa5134ee26d2 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:42.005314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c542-f0e7-4f2f-b90f-fa5134ee26d2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:42.005314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9c542-f0e7-4f2f-b90f-fa5134ee26d2 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:42.006734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 27 08:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:37:42.012224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf1cdf-1575-4f64-a5f3-cd2731edd338 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:42.012639 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf1cdf-1575-4f64-a5f3-cd2731edd338 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:42.012779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf1cdf-1575-4f64-a5f3-cd2731edd338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:42.032525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9faf1cdf-1575-4f64-a5f3-cd2731edd338 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:37:42.033725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9faf1cdf-1575-4f64-a5f3-cd2731edd338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:42.041478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 27 08:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:37:42.345293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9c542-f0e7-4f2f-b90f-fa5134ee26d2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:37:42.347036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f8f9c542-f0e7-4f2f-b90f-fa5134ee26d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:37:42.354546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=77 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:37:42.360299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9c542-f0e7-4f2f-b90f-fa5134ee26d2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:37:42.363785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5456a-9b10-4b5d-91fa-4df320097b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:37:42.364385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5456a-9b10-4b5d-91fa-4df320097b3f service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:37:42.369047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5456a-9b10-4b5d-91fa-4df320097b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:37:42.369047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5456a-9b10-4b5d-91fa-4df320097b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:37:42.374493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5456a-9b10-4b5d-91fa-4df320097b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:37:42.375971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5456a-9b10-4b5d-91fa-4df320097b3f service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:37:42.385540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 775/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:37:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:37:52.026982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=47 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:37:52.404816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:14.087434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef649-2f13-497a-b513-3ed295150221 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.088108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef649-2f13-497a-b513-3ed295150221 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.088108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ef649-2f13-497a-b513-3ed295150221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.124106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-366ef649-2f13-497a-b513-3ed295150221 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.125322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-366ef649-2f13-497a-b513-3ed295150221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.133770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 765/1540] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:39:14.142981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d4a3c-06b7-41bc-b99e-ef109a01aa28 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.143545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d4a3c-06b7-41bc-b99e-ef109a01aa28 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.143545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d4a3c-06b7-41bc-b99e-ef109a01aa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.172285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c44d4a3c-06b7-41bc-b99e-ef109a01aa28 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.173483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c44d4a3c-06b7-41bc-b99e-ef109a01aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.185635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 776/1541] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.190600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef649-2f13-497a-b513-3ed295150221 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.205084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ef649-2f13-497a-b513-3ed295150221 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.210892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3e821-2731-4f8f-ae32-bc870343439e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.211754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e821-2731-4f8f-ae32-bc870343439e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.212038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e821-2731-4f8f-ae32-bc870343439e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.212317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3e821-2731-4f8f-ae32-bc870343439e admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.213021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3e821-2731-4f8f-ae32-bc870343439e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:14.213325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3e821-2731-4f8f-ae32-bc870343439e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.214964 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3e821-2731-4f8f-ae32-bc870343439e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.234010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 766/1542] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:14.242166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d4a3c-06b7-41bc-b99e-ef109a01aa28 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.242857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0b52-d294-4322-9a8c-05b21f753ce5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.243146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0b52-d294-4322-9a8c-05b21f753ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.243425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0b52-d294-4322-9a8c-05b21f753ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.261064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d4a3c-06b7-41bc-b99e-ef109a01aa28 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.268909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6b8f0b52-d294-4322-9a8c-05b21f753ce5 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.270118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6b8f0b52-d294-4322-9a8c-05b21f753ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.272655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a90ef-6a5c-4181-a6e0-f5b91f75be63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.273381 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a90ef-6a5c-4181-a6e0-f5b91f75be63 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.273666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a90ef-6a5c-4181-a6e0-f5b91f75be63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.273941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a90ef-6a5c-4181-a6e0-f5b91f75be63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.274757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a90ef-6a5c-4181-a6e0-f5b91f75be63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.276674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a90ef-6a5c-4181-a6e0-f5b91f75be63 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.286323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 767/1543] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.296288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8254-a9d4-44e2-89b1-6487790dbbcc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.296607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8254-a9d4-44e2-89b1-6487790dbbcc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.296843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8254-a9d4-44e2-89b1-6487790dbbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.327937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8ddf8254-a9d4-44e2-89b1-6487790dbbcc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.329583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8ddf8254-a9d4-44e2-89b1-6487790dbbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.342823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 768/1544] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:39:14.343796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea724c5b-f97f-44ad-8a32-73e2a7a5f001 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.344085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea724c5b-f97f-44ad-8a32-73e2a7a5f001 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.344357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea724c5b-f97f-44ad-8a32-73e2a7a5f001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.371676 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ea724c5b-f97f-44ad-8a32-73e2a7a5f001 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.372945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ea724c5b-f97f-44ad-8a32-73e2a7a5f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.386927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 769/1545] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.387870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea724c5b-f97f-44ad-8a32-73e2a7a5f001 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.402583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea724c5b-f97f-44ad-8a32-73e2a7a5f001 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.407794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73f530-e27f-4bf6-887a-d00f31c23b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.408420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f530-e27f-4bf6-887a-d00f31c23b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.408700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f530-e27f-4bf6-887a-d00f31c23b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.408977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f530-e27f-4bf6-887a-d00f31c23b0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.409679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73f530-e27f-4bf6-887a-d00f31c23b0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:14.409990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73f530-e27f-4bf6-887a-d00f31c23b0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.411614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73f530-e27f-4bf6-887a-d00f31c23b0a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.415412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 777/1546] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:14.417067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4456b-360c-4a86-9a08-4cd919b4d89e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.417358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4456b-360c-4a86-9a08-4cd919b4d89e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.427580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4456b-360c-4a86-9a08-4cd919b4d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.440086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 770/1547] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:14.443327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656efe72-2c90-43c2-8d62-baed161b76b9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.444269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656efe72-2c90-43c2-8d62-baed161b76b9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.444652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656efe72-2c90-43c2-8d62-baed161b76b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.460177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d5f4456b-360c-4a86-9a08-4cd919b4d89e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.461400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d5f4456b-360c-4a86-9a08-4cd919b4d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.477548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-656efe72-2c90-43c2-8d62-baed161b76b9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.478790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-656efe72-2c90-43c2-8d62-baed161b76b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.532236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 778/1548] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.533144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4456b-360c-4a86-9a08-4cd919b4d89e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.543347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4456b-360c-4a86-9a08-4cd919b4d89e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.547853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f2a7f-2f8d-4e4d-a67c-f664e17a3ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.548679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 771/1549] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.549999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edcc84-d0b0-426b-8b61-50b1ce71682d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.550333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edcc84-d0b0-426b-8b61-50b1ce71682d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.550608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edcc84-d0b0-426b-8b61-50b1ce71682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.554988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f2a7f-2f8d-4e4d-a67c-f664e17a3ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.555442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f2a7f-2f8d-4e4d-a67c-f664e17a3ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.555989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f2a7f-2f8d-4e4d-a67c-f664e17a3ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.556672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f2a7f-2f8d-4e4d-a67c-f664e17a3ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.557712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING py.warnings [None req-167f2a7f-2f8d-4e4d-a67c-f664e17a3ba2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:39:14.557712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: warnings.warn(msg) Jul 27 08:39:14.557712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: Jul 27 08:39:14.558983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f2a7f-2f8d-4e4d-a67c-f664e17a3ba2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.570743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-75edcc84-d0b0-426b-8b61-50b1ce71682d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.572060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-75edcc84-d0b0-426b-8b61-50b1ce71682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.579880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 772/1550] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.580773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edcc84-d0b0-426b-8b61-50b1ce71682d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.590600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edcc84-d0b0-426b-8b61-50b1ce71682d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.595269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.595788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.596064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.596332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.596975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.597976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:39:14.597976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:39:14.597976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:39:14.599235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.633340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 779/1551] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:14.633340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb78ef3-b436-49d4-b160-6641f47558da None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.633340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb78ef3-b436-49d4-b160-6641f47558da None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.633340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb78ef3-b436-49d4-b160-6641f47558da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.651635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-deb78ef3-b436-49d4-b160-6641f47558da None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.652832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-deb78ef3-b436-49d4-b160-6641f47558da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.660635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 780/1552] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.661757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53721-ba4e-4848-82c8-903dccb4e6b6 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.662038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53721-ba4e-4848-82c8-903dccb4e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.662853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e53721-ba4e-4848-82c8-903dccb4e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.681772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f9e53721-ba4e-4848-82c8-903dccb4e6b6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.683090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f9e53721-ba4e-4848-82c8-903dccb4e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.690856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 781/1553] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.692132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9227f-2462-4692-9fb3-cdfb0efc6727 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.692406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9227f-2462-4692-9fb3-cdfb0efc6727 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.692672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9227f-2462-4692-9fb3-cdfb0efc6727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.711909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c0a9227f-2462-4692-9fb3-cdfb0efc6727 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.716786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 773/1554] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:14.716925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c0a9227f-2462-4692-9fb3-cdfb0efc6727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.717037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.773734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 782/1555] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.775040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd972f-9105-46e5-b87e-2c739df13ca4 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.775330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd972f-9105-46e5-b87e-2c739df13ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.775604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd972f-9105-46e5-b87e-2c739df13ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.780728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98833678-0b9e-4a4c-9677-71f77bc7d42a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.785373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858263b-71a3-4e72-8c54-0d63ca0a6537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.786055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858263b-71a3-4e72-8c54-0d63ca0a6537 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.786412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858263b-71a3-4e72-8c54-0d63ca0a6537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.793097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858263b-71a3-4e72-8c54-0d63ca0a6537 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.793097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858263b-71a3-4e72-8c54-0d63ca0a6537 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:14.793097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858263b-71a3-4e72-8c54-0d63ca0a6537 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.793097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858263b-71a3-4e72-8c54-0d63ca0a6537 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.798264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d5bd972f-9105-46e5-b87e-2c739df13ca4 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.799482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d5bd972f-9105-46e5-b87e-2c739df13ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.808191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 774/1556] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:14.808191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053677f-c25d-4be4-85da-b1abb8bb7d91 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.808191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053677f-c25d-4be4-85da-b1abb8bb7d91 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.808191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053677f-c25d-4be4-85da-b1abb8bb7d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.813023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 783/1557] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.813860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd972f-9105-46e5-b87e-2c739df13ca4 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.823910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd972f-9105-46e5-b87e-2c739df13ca4 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.828044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3053677f-c25d-4be4-85da-b1abb8bb7d91 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.828460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47fb35-589f-4d60-b48d-54688cbcf6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.829067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47fb35-589f-4d60-b48d-54688cbcf6af admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.829235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3053677f-c25d-4be4-85da-b1abb8bb7d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.829351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47fb35-589f-4d60-b48d-54688cbcf6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.829604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47fb35-589f-4d60-b48d-54688cbcf6af admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.830329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47fb35-589f-4d60-b48d-54688cbcf6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.831927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47fb35-589f-4d60-b48d-54688cbcf6af admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.837736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 775/1558] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.839092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69340f-208c-43b9-b53d-14c885e44d99 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.839370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69340f-208c-43b9-b53d-14c885e44d99 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.839642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69340f-208c-43b9-b53d-14c885e44d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.846044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 784/1559] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:14.846044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae4c2c-e0c9-4c4c-a516-aef1501b0993 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.846044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae4c2c-e0c9-4c4c-a516-aef1501b0993 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.846044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae4c2c-e0c9-4c4c-a516-aef1501b0993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.856158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:14.861968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3a69340f-208c-43b9-b53d-14c885e44d99 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.867013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3a69340f-208c-43b9-b53d-14c885e44d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.867124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-49ae4c2c-e0c9-4c4c-a516-aef1501b0993 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.867124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-49ae4c2c-e0c9-4c4c-a516-aef1501b0993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.871833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 776/1560] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.872657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69340f-208c-43b9-b53d-14c885e44d99 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.874608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 785/1561] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.875657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e2f20-2841-42d9-ba72-b70ca0d120c0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.876150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e2f20-2841-42d9-ba72-b70ca0d120c0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.876455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e2f20-2841-42d9-ba72-b70ca0d120c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.883197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69340f-208c-43b9-b53d-14c885e44d99 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.887956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.888566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.888840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.889131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.889833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.891877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.895205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-000e2f20-2841-42d9-ba72-b70ca0d120c0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:14.896387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-000e2f20-2841-42d9-ba72-b70ca0d120c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:14.904452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 786/1562] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:14.907287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e2f20-2841-42d9-ba72-b70ca0d120c0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:14.915440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e2f20-2841-42d9-ba72-b70ca0d120c0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:14.920076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:14.920706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:14.920988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:14.921261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:14.921942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:14.923965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:14.933862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:14.999032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 777/1563] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:14.999032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.006486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ada45e-0cb8-47aa-b18e-b379d1d6c5bd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.012224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 787/1564] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:14 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.012224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.014794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.015406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.015685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.015955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.016644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:15.016938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.018642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.031188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 778/1565] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:15.031988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.042710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76a62b-93bd-4eb1-8ca0-e110b53fd677 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.073507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec7c71-4140-4a6b-b560-c4ca95411bfc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.078032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.078682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.078958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.079232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.079921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:15.080221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.081261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING py.warnings [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:39:15.081261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: warnings.warn(msg) Jul 27 08:39:15.081261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: Jul 27 08:39:15.082137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.094446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 788/1566] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:15.095237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.105001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.105679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.106027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.106340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.107032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.108611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.109404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba379503-4038-4e9c-8c80-4bddf96047a1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.114114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.114753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.115037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.115342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.116094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:15.116403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.117981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.119352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 779/1567] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:15.120156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.129387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007e9be-0b41-4fac-acc8-549a80f664e1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.133737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 789/1568] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:15.134478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.134611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.135217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.135475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.135750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.136552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:15.136859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.138457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.146309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309579a-2671-4a91-b5c3-726fe587397a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.151514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 780/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:15.152239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.156586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.157520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.157808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.158683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.159398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:15.159868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.161527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.161790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eeb6c-cc85-45d8-b49c-16c98acc6fca admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.166656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.167254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.167536 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.167812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.168495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.170062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.180267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 790/1570] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:15.180267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.185100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e0ce2-0b94-4bcc-b690-782851c38e88 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.186333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 781/1571] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:15.187056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.191103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.191882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.192172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.192568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.193244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.196077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.197145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b474a-c4ae-4218-97a0-be5139936d00 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.201727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.202350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.202634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.202911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.203628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.205527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.305369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 791/1572] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/users => generated 286 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.306310 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.316177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f740d-1552-4b9d-93a9-80b1f84a40aa admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.321212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.324896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 782/1573] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.325044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.325044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.325044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.325325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.325433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.325433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.387566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b08799-03c8-45e7-9051-c8d5018cc6bf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.392469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.393136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.393413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.393690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] PATH_INFO: `/v3/projects/bbedb0ad0538471e98b0641598f08c17/users/37554602ef6d461b97660ad6b337ed98/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.408928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbedb0ad0538471e98b0641598f08c17, user_id=37554602ef6d461b97660ad6b337ed98, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.410269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:39:15.410269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:39:15.410269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:39:15.412232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.421751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 792/1574] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.423055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.485288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea741f-3b8a-4f28-bb6d-d117eebd10c5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.489915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.491179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.491663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.492019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.492681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.494631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.503241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 783/1575] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:15 2022] PUT /identity/v3/projects/bbedb0ad0538471e98b0641598f08c17/users/37554602ef6d461b97660ad6b337ed98/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:15.506435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.512500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:15.569689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e418542-ed4e-427e-af21-f987b0d867c7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.574314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.574921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.575197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.575465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.576166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.578136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.592079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 793/1576] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.593415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.597853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:15.655599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df895e75-992b-4d1c-bc7f-498d5f899fd2 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.660186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.660769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.661045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.661314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.662013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.664629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.677089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 784/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.678581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.738212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad7e58-6878-42a9-a56f-f3e0537d5bf3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.744269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.744871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.745836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.745836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.746081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.748341 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.759201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 785/1578] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:15.760015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.766681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 794/1579] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.766681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.772190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0746304-6565-48fe-8607-009120eb028d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.829296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.829734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dccf77-9efb-41dd-bc56-dd07f9b06e4d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.829908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.830216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.830611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.831320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.833143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.834958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc8838-6baf-45dc-9c52-11ab311a999c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.835568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc8838-6baf-45dc-9c52-11ab311a999c admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.835852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc8838-6baf-45dc-9c52-11ab311a999c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.836121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc8838-6baf-45dc-9c52-11ab311a999c admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.836778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc8838-6baf-45dc-9c52-11ab311a999c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.838725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc8838-6baf-45dc-9c52-11ab311a999c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.843046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 786/1580] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:15.843816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.853370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d74e8-bb4d-4c92-b2a8-d4e25112df0b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.857927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.860279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.860279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.860279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.860279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.861724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.932664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 795/1581] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.933837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f22b1-0863-4171-bbb8-c1f1d146b7a2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.934124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f22b1-0863-4171-bbb8-c1f1d146b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.934422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f22b1-0863-4171-bbb8-c1f1d146b7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.953420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fc7f22b1-0863-4171-bbb8-c1f1d146b7a2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:15.954650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fc7f22b1-0863-4171-bbb8-c1f1d146b7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:15.962728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 796/1582] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:15.963469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f22b1-0863-4171-bbb8-c1f1d146b7a2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.967929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 787/1583] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:15.967929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:15.972626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:15.977665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a4a33-0d38-4d0d-9eed-75bc00e7cf2f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.981314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f22b1-0863-4171-bbb8-c1f1d146b7a2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:15.982298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.982911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.983194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.983469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.984130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.985902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:15.986076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:15.986555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:15.986831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:15.987102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:15.988085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:15.990440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.076118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 788/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:16.076498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 797/1585] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:15 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:16.077111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.077875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.088328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c2f71-f292-4dbb-ba30-6059d81c363d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.088717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6daec-3aeb-4304-baa2-ae152c7818b1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.093162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.093338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.093762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.093986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.094090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.094289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.094399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.094554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] PATH_INFO: `/v3/projects/9dcfebb6e1774a7daed20b60d2379875/users/c29a83c5d2ce48eda587f2cec0e959ce/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.094989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.096945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.110665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dcfebb6e1774a7daed20b60d2379875, user_id=c29a83c5d2ce48eda587f2cec0e959ce, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.113335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.207863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 798/1586] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:16 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:16.208624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.218437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024d9be-cc65-49b7-8f76-a193caf19a2e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.219035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 789/1587] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:16 2022] PUT /identity/v3/projects/9dcfebb6e1774a7daed20b60d2379875/users/c29a83c5d2ce48eda587f2cec0e959ce/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:16.219859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.254807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:16.280019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.280665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.280948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.281224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.282041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.282691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c893ce0-487f-4d10-8c88-5c77fa24930e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.283706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.287310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.288458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.288693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.288961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.290350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.291290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.294734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 799/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.295554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.305218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 790/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.305218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.309982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6abd8-1758-439e-854b-cc6ec88b22ba admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.314827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.315437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.315735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.315981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.316668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.318348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.329268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 800/1590] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.330068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.341515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e1e43-4548-429d-ace3-d2922f3a83ff admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.346087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.346717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.346997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.347271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.347953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.349567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.360260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 801/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.360406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f948b-4d10-4931-918f-3e59a786dfec admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.361052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.364281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.364886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.365165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.365434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.371240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ead50-bab4-4e40-b0a7-aa7be2838dde admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.375781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.375938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00165c-f26f-45c7-9172-ca0dc14d3c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.376663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00165c-f26f-45c7-9172-ca0dc14d3c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.376793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00165c-f26f-45c7-9172-ca0dc14d3c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.377028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00165c-f26f-45c7-9172-ca0dc14d3c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.377721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00165c-f26f-45c7-9172-ca0dc14d3c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.377959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.379327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00165c-f26f-45c7-9172-ca0dc14d3c81 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.388824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 791/1592] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:16.389928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.393609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 802/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.393609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7e277-f62b-496b-8c14-9ede5902b44e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.393609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7e277-f62b-496b-8c14-9ede5902b44e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.393609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7e277-f62b-496b-8c14-9ede5902b44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.401139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf86538-7542-49a0-b20a-f493dd616ade service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.406353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.407226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.407576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.407929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.408629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.410281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.411977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e4d7e277-f62b-496b-8c14-9ede5902b44e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:16.413752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e4d7e277-f62b-496b-8c14-9ede5902b44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:16.424235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 792/1594] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.424423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 803/1595] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:16.424538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.424633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7e277-f62b-496b-8c14-9ede5902b44e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.434794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7e277-f62b-496b-8c14-9ede5902b44e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.436694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5c046-36c0-428e-bbd0-5e58fe4a86a3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.439679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.440062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.440362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.440618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.441311 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.441432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.442027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.442340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.442612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] PATH_INFO: `/v3/projects/d3cea0410a2f4fe88f5f300db0cabbe1/users/6e3f9e581f0a4d64be395f54a5432e7d/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.442946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.456893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 804/1596] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.456893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.458137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cea0410a2f4fe88f5f300db0cabbe1, user_id=6e3f9e581f0a4d64be395f54a5432e7d, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.461838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.466910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ca683-3a55-4584-8c94-b2ea8bdf5080 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.471542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.472130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.472406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.472675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.473424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.476623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.487729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 805/1597] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.488646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.498544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222645fe-1035-40bc-931b-a7b3f5dd3cbd admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.503137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.503707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.503985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.504267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.504928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.506963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.517211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 806/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.518085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.530286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bfb2c-723a-4b2c-bb18-7ad98887a1f8 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.534825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.535429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.535712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.535987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.536669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.539145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.546182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 793/1599] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:16 2022] PUT /identity/v3/projects/d3cea0410a2f4fe88f5f300db0cabbe1/users/6e3f9e581f0a4d64be395f54a5432e7d/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:16.559312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.559471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 807/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.559471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.574616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855b7cc-f89c-4362-9630-e588a305c806 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.597107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:16.628649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.629354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.629701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.630045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.632540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438bfef1-ce7d-4894-865d-cd29f171d2e3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.637088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.637710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.637992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.638300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.638997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.639476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.640629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.641015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.651054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 794/1601] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:16.651599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 808/1602] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:16.651873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.652436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.662239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c035-124b-439f-9fda-6be4801d2dcc service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.666442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96f824-f1e6-4b18-90d9-96576ee8717f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.668366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.668717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.669025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.669300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.669567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] PATH_INFO: `/v3/projects/e2c60f49a9e14417bd1d9504128fc94b/users/26128a80933f4be3bed504d7a2d2785f/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.669674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.669975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.670367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] PATH_INFO: `/v3/projects/3d331b98cf004966a50f39a5fd75a856/users/84ad6c639c42426381dc972ea8527f13/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.686096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d331b98cf004966a50f39a5fd75a856, user_id=84ad6c639c42426381dc972ea8527f13, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.686328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c60f49a9e14417bd1d9504128fc94b, user_id=26128a80933f4be3bed504d7a2d2785f, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.688808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.689024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.829817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 795/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:16 2022] PUT /identity/v3/projects/3d331b98cf004966a50f39a5fd75a856/users/84ad6c639c42426381dc972ea8527f13/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:16.832735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.833205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 809/1604] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:16 2022] PUT /identity/v3/projects/e2c60f49a9e14417bd1d9504128fc94b/users/26128a80933f4be3bed504d7a2d2785f/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:16.834285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:16.874655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:16.909956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89173ba3-eabf-4609-a3f4-050bab26aaa4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.910640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8f0f3-3153-442d-b01b-89bdad710b78 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:16.915028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399aec01-a15d-4f4e-a310-b89ba1c9c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.915686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aec01-a15d-4f4e-a310-b89ba1c9c082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.915964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aec01-a15d-4f4e-a310-b89ba1c9c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.916236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399aec01-a15d-4f4e-a310-b89ba1c9c082 admin admin] PATH_INFO: `/v3/projects/173b06cf43cd421eb02bfdfcffcc6a01/users/5fcb4c0e6c60442da43fa46daf41345c/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.919661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:16.920371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:16.920679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:16.920997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] PATH_INFO: `/v3/projects/47dc670d30af4db294d3b0a6922e3d8e/users/fdcd322e1e404105a230cbe7528eb0bc/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:16.935006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aec01-a15d-4f4e-a310-b89ba1c9c082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173b06cf43cd421eb02bfdfcffcc6a01, user_id=5fcb4c0e6c60442da43fa46daf41345c, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.937410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=47dc670d30af4db294d3b0a6922e3d8e, user_id=fdcd322e1e404105a230cbe7528eb0bc, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:16.938256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399aec01-a15d-4f4e-a310-b89ba1c9c082 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:16.940225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:17.001494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 810/1605] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:16 2022] PUT /identity/v3/projects/173b06cf43cd421eb02bfdfcffcc6a01/users/5fcb4c0e6c60442da43fa46daf41345c/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:17.002890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460f69a-992f-4d7a-80ad-1a179774ba8b None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.009012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460f69a-992f-4d7a-80ad-1a179774ba8b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.009012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460f69a-992f-4d7a-80ad-1a179774ba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.015585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 796/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:16 2022] PUT /identity/v3/projects/47dc670d30af4db294d3b0a6922e3d8e/users/fdcd322e1e404105a230cbe7528eb0bc/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:17.016651 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:17.064673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f460f69a-992f-4d7a-80ad-1a179774ba8b None None] MFA Rules not processed for user `6e3f9e581f0a4d64be395f54a5432e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.065909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f460f69a-992f-4d7a-80ad-1a179774ba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.123714 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd2823-3138-488d-bb9b-109afe2344cd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:17.129228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f30b10-9479-47ac-9d05-a7546b3ad0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:17.130104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f30b10-9479-47ac-9d05-a7546b3ad0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.130415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f30b10-9479-47ac-9d05-a7546b3ad0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.130685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f30b10-9479-47ac-9d05-a7546b3ad0f9 admin admin] PATH_INFO: `/v3/projects/940250de2f6f47ed9edc1bb0468551c8/users/c370dd8fa3aa407ca22cce8425fcbc25/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.146763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f30b10-9479-47ac-9d05-a7546b3ad0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940250de2f6f47ed9edc1bb0468551c8, user_id=c370dd8fa3aa407ca22cce8425fcbc25, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:17.149580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f30b10-9479-47ac-9d05-a7546b3ad0f9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:17.150083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 811/1607] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.151413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ca0fc-a588-437d-a035-a9404174a0f8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.151818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ca0fc-a588-437d-a035-a9404174a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.152101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ca0fc-a588-437d-a035-a9404174a0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.170745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ef5ca0fc-a588-437d-a035-a9404174a0f8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.171939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ef5ca0fc-a588-437d-a035-a9404174a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.180901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 812/1608] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.182168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96603bbd-ffd5-43f9-8977-44fc945248ba None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.182680 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96603bbd-ffd5-43f9-8977-44fc945248ba None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.182847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96603bbd-ffd5-43f9-8977-44fc945248ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.214637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-96603bbd-ffd5-43f9-8977-44fc945248ba None None] MFA Rules not processed for user `84ad6c639c42426381dc972ea8527f13`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.214637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-96603bbd-ffd5-43f9-8977-44fc945248ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.225986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 797/1609] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:17 2022] PUT /identity/v3/projects/940250de2f6f47ed9edc1bb0468551c8/users/c370dd8fa3aa407ca22cce8425fcbc25/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:17.229838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75618eb-ebc4-4648-ac9b-49b7e72ca960 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.229838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75618eb-ebc4-4648-ac9b-49b7e72ca960 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.229838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75618eb-ebc4-4648-ac9b-49b7e72ca960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.238164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:17.281462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d75618eb-ebc4-4648-ac9b-49b7e72ca960 None None] MFA Rules not processed for user `5fcb4c0e6c60442da43fa46daf41345c`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.283596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d75618eb-ebc4-4648-ac9b-49b7e72ca960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.321833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 813/1610] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.324297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.325353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.325994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.363677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] MFA Rules not processed for user `fdcd322e1e404105a230cbe7528eb0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.364888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.375204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 798/1611] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.381460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75618eb-ebc4-4648-ac9b-49b7e72ca960 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:17.381965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:17.448977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 814/1612] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.449986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:17.451289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.453005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75618eb-ebc4-4648-ac9b-49b7e72ca960 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:17.457851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:17.458680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.459138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.459507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.460949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:17.463647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:17.516657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d7165-5df0-45a9-9d73-0bdfeaa987e5 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:17.520745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56d500-a528-40c9-ac42-8fcab98511ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:17.526483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56d500-a528-40c9-ac42-8fcab98511ba service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.526795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56d500-a528-40c9-ac42-8fcab98511ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.527084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56d500-a528-40c9-ac42-8fcab98511ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.539164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56d500-a528-40c9-ac42-8fcab98511ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:17.540863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56d500-a528-40c9-ac42-8fcab98511ba service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:17.551913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 815/1613] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:17.553136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56ae57-f8ef-4517-a3e4-cb0d172ef689 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.561632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56ae57-f8ef-4517-a3e4-cb0d172ef689 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.561632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56ae57-f8ef-4517-a3e4-cb0d172ef689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.576323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-8d56ae57-f8ef-4517-a3e4-cb0d172ef689 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.577758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8d56ae57-f8ef-4517-a3e4-cb0d172ef689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.587926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 799/1614] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:17.596023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 816/1615] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.596023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9a7d1-149a-4df6-9c51-22f9fb212b11 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.596023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9a7d1-149a-4df6-9c51-22f9fb212b11 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.596023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9a7d1-149a-4df6-9c51-22f9fb212b11 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.598588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 817/1616] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 27 08:39:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:17.600025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b87a9-442e-4443-8576-2a025d2fe977 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.600464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b87a9-442e-4443-8576-2a025d2fe977 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.601581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b87a9-442e-4443-8576-2a025d2fe977 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.603267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 818/1617] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 27 08:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:17.604519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b87a9-442e-4443-8576-2a025d2fe977 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:17.606617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:17.680383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b87a9-442e-4443-8576-2a025d2fe977 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:17.685155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:17.685778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.687787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.687889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ed07-f545-49f9-ac29-7ae9283f200d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:17.687974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.687974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:17.688928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:17.691968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3945a-fc4f-44a0-892e-496147ba1b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:17.692586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3945a-fc4f-44a0-892e-496147ba1b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.692864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3945a-fc4f-44a0-892e-496147ba1b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.693139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3945a-fc4f-44a0-892e-496147ba1b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.693935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3945a-fc4f-44a0-892e-496147ba1b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:17.696757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3945a-fc4f-44a0-892e-496147ba1b79 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:17.765700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 819/1618] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:17.765700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:17.794662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 800/1619] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/projects => generated 367 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:17.794662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602e821-5ce6-43ea-968e-723ac637bb0d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.794662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602e821-5ce6-43ea-968e-723ac637bb0d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.794662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602e821-5ce6-43ea-968e-723ac637bb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.810859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:17.830052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d1751-e34b-42f8-ac74-c86f51044f8e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:17.847004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5602e821-5ce6-43ea-968e-723ac637bb0d None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.850218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5602e821-5ce6-43ea-968e-723ac637bb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.901298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:17.902111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.902424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.902699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.920057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 801/1620] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.921543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95bab-8fbd-442c-a5b3-ae25f4e21f53 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:17.921931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95bab-8fbd-442c-a5b3-ae25f4e21f53 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:17.922345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb95bab-8fbd-442c-a5b3-ae25f4e21f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:17.949648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bfb95bab-8fbd-442c-a5b3-ae25f4e21f53 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:17.951302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bfb95bab-8fbd-442c-a5b3-ae25f4e21f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:17.964110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 802/1621] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 27 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:17.964604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95bab-8fbd-442c-a5b3-ae25f4e21f53 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.013239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.014904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.028078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 820/1622] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:18.028937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.040270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5213064-6046-4587-b09d-72760d2bb136 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.047632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.048312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.048599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.048873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.049537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.051711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.077567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb95bab-8fbd-442c-a5b3-ae25f4e21f53 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.083988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.084911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.085704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.085859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.086700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.088503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.102197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 803/1623] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:18.102197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.118414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbb08a-317b-4969-a1e3-e280071e4f8e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.128301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.128301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.128301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.128301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.205980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.208505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.220243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 821/1624] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:18 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:18.220243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.225614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 804/1625] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:18.226641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.239103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9269f-34d5-469c-81f5-6a6670537b9d service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.246603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8942a6-e103-49d6-a57b-b1fa16ea1dce admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.247705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.249380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.249775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.251940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] PATH_INFO: `/v3/projects/4bf6a3cd5d3545118bf86578de657005/users/f3d1ddd8c35047f8ab09afe70c6819c9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.264184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.264755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.265037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.265312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.273541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf6a3cd5d3545118bf86578de657005, user_id=f3d1ddd8c35047f8ab09afe70c6819c9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.277776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.362238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.367224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.389374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 822/1626] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:18.392093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.400600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 805/1627] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:18 2022] PUT /identity/v3/projects/4bf6a3cd5d3545118bf86578de657005/users/f3d1ddd8c35047f8ab09afe70c6819c9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:18.401710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.415366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d9ef2-d275-4f1f-950f-d66fb7964c23 service nova] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.432328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:18.558318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5871e5-368b-453c-a67c-e8c9fe522280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.558964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5871e5-368b-453c-a67c-e8c9fe522280 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.559249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5871e5-368b-453c-a67c-e8c9fe522280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.559524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5871e5-368b-453c-a67c-e8c9fe522280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.560594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5871e5-368b-453c-a67c-e8c9fe522280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.561146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fee4e-6f5e-4f12-b784-b1d20ee00c09 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.562313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5871e5-368b-453c-a67c-e8c9fe522280 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.579899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 823/1628] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:18.580106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08deeb-a2ed-4dd6-a39f-9be8d51353c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.580755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08deeb-a2ed-4dd6-a39f-9be8d51353c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.581036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08deeb-a2ed-4dd6-a39f-9be8d51353c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.581284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce520-9337-4dfe-8e76-c7271b982b62 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.581396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08deeb-a2ed-4dd6-a39f-9be8d51353c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.581579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce520-9337-4dfe-8e76-c7271b982b62 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.581918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce520-9337-4dfe-8e76-c7271b982b62 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.582220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08deeb-a2ed-4dd6-a39f-9be8d51353c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.584048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08deeb-a2ed-4dd6-a39f-9be8d51353c2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.589991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 824/1629] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 27 08:39:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:18.590943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ce520-9337-4dfe-8e76-c7271b982b62 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.606031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ce520-9337-4dfe-8e76-c7271b982b62 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.607697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 806/1630] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:18.611144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.611588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da452270-d0c5-4c63-94a5-de7fd6f9508d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.611849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.611968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da452270-d0c5-4c63-94a5-de7fd6f9508d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.612145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da452270-d0c5-4c63-94a5-de7fd6f9508d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.612262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.612508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] PATH_INFO: `/v3/projects/88b260b655614415a0cf3a586035a717/users/a5b0e464b89f40569d9f23879e483835/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.630293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b260b655614415a0cf3a586035a717, user_id=a5b0e464b89f40569d9f23879e483835, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.636174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.653754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-da452270-d0c5-4c63-94a5-de7fd6f9508d None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:18.655251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-da452270-d0c5-4c63-94a5-de7fd6f9508d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:18.720753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 825/1631] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:18 2022] PUT /identity/v3/projects/88b260b655614415a0cf3a586035a717/users/a5b0e464b89f40569d9f23879e483835/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:18.721674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.766912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 807/1632] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:18.768052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da452270-d0c5-4c63-94a5-de7fd6f9508d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.796171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da452270-d0c5-4c63-94a5-de7fd6f9508d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.825158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84236b2f-1532-4ca5-83d9-cf1a3355a4c6 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.831333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.832549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.833146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.838135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] PATH_INFO: `/v3/projects/4bf6a3cd5d3545118bf86578de657005/users/f3d1ddd8c35047f8ab09afe70c6819c9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.838135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf6a3cd5d3545118bf86578de657005, user_id=f3d1ddd8c35047f8ab09afe70c6819c9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.839664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.858219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 826/1633] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:18 2022] PUT /identity/v3/projects/4bf6a3cd5d3545118bf86578de657005/users/f3d1ddd8c35047f8ab09afe70c6819c9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:18.858219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:18.905374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfb94a-372a-4f42-a6cf-ecedf44c110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.907719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfb94a-372a-4f42-a6cf-ecedf44c110d admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.913974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfb94a-372a-4f42-a6cf-ecedf44c110d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.913974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfb94a-372a-4f42-a6cf-ecedf44c110d admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.913974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfb94a-372a-4f42-a6cf-ecedf44c110d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:18.916624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfb94a-372a-4f42-a6cf-ecedf44c110d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:18.937008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 808/1634] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:18.942378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525826-e66e-43d0-8319-a60cc6db2928 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.944687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525826-e66e-43d0-8319-a60cc6db2928 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.944687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525826-e66e-43d0-8319-a60cc6db2928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.964078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b936aa0-0130-4379-b17b-ec5ed9e97edf admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:18.968187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:18.971854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c8de2-1371-40fb-8557-3eb6c16cbddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:18.971854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c8de2-1371-40fb-8557-3eb6c16cbddb service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:18.971854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c8de2-1371-40fb-8557-3eb6c16cbddb service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:18.971854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c8de2-1371-40fb-8557-3eb6c16cbddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:18.993436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-55525826-e66e-43d0-8319-a60cc6db2928 None None] MFA Rules not processed for user `f3d1ddd8c35047f8ab09afe70c6819c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:18.994518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-55525826-e66e-43d0-8319-a60cc6db2928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:19.106297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 809/1635] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:19.110534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cf26e-9241-4bf0-8e69-5c09bdbaa766 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:19.110833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cf26e-9241-4bf0-8e69-5c09bdbaa766 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:19.112568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cf26e-9241-4bf0-8e69-5c09bdbaa766 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:19.120852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 810/1636] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 27 08:39:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:19.120852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cf26e-9241-4bf0-8e69-5c09bdbaa766 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:19.121072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c8de2-1371-40fb-8557-3eb6c16cbddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:19.121072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c8de2-1371-40fb-8557-3eb6c16cbddb service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:19.144019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 827/1637] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:19.145403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40245b-c5d5-4ea8-bd60-9f55ef37b4a5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:19.145732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40245b-c5d5-4ea8-bd60-9f55ef37b4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:19.145985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40245b-c5d5-4ea8-bd60-9f55ef37b4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:19.191254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ad40245b-c5d5-4ea8-bd60-9f55ef37b4a5 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:19.192485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ad40245b-c5d5-4ea8-bd60-9f55ef37b4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:19.215083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 828/1638] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:19.216833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40245b-c5d5-4ea8-bd60-9f55ef37b4a5 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:19.224878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cf26e-9241-4bf0-8e69-5c09bdbaa766 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:19.232653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40245b-c5d5-4ea8-bd60-9f55ef37b4a5 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:19.239415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:19.244784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:19.244784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:19.244784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:19.247624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da1e05-9653-4404-8e21-38f55adaaa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:19.248398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da1e05-9653-4404-8e21-38f55adaaa35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:19.248763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da1e05-9653-4404-8e21-38f55adaaa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:19.254645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da1e05-9653-4404-8e21-38f55adaaa35 admin admin] PATH_INFO: `/v3/projects/88b260b655614415a0cf3a586035a717/users/a5b0e464b89f40569d9f23879e483835/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:19.258049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da1e05-9653-4404-8e21-38f55adaaa35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b260b655614415a0cf3a586035a717, user_id=a5b0e464b89f40569d9f23879e483835, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:19.261542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da1e05-9653-4404-8e21-38f55adaaa35 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:19.261784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:19.269008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:19.294130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 811/1639] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:19 2022] PUT /identity/v3/projects/88b260b655614415a0cf3a586035a717/users/a5b0e464b89f40569d9f23879e483835/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:19.305012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d468b-09df-4f11-b10b-371007437d7b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:19.307986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d468b-09df-4f11-b10b-371007437d7b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:19.307986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d468b-09df-4f11-b10b-371007437d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:19.314097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 829/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:19.327568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:19.373667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b66d468b-09df-4f11-b10b-371007437d7b None None] MFA Rules not processed for user `a5b0e464b89f40569d9f23879e483835`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:19.373667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b66d468b-09df-4f11-b10b-371007437d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:19.432654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:19.492314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 812/1641] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:19.557322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:19.567840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6965384-0bbb-4c9b-b69c-dada022db393 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:19.663763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:19.664614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:19.665125 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:19.665921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:19.799549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:19.801297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:19.815650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 830/1642] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:20.357740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d468b-09df-4f11-b10b-371007437d7b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:20.372043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d468b-09df-4f11-b10b-371007437d7b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:20.380534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967fdb-e333-490d-bffc-0d18f3f23d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:20.381343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967fdb-e333-490d-bffc-0d18f3f23d98 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:20.382947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967fdb-e333-490d-bffc-0d18f3f23d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:20.383221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967fdb-e333-490d-bffc-0d18f3f23d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:20.395477 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14967fdb-e333-490d-bffc-0d18f3f23d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:20.397313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14967fdb-e333-490d-bffc-0d18f3f23d98 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:20.422918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 813/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:21.076348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:21.235449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:22.750001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:22.881020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:22.904077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82802828-375c-4d83-aa21-923f762be399 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:22.913942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:22.915957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:22.916347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:22.916783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] PATH_INFO: `/v3/users/6e3f9e581f0a4d64be395f54a5432e7d` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:22.918446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3f9e581f0a4d64be395f54a5432e7d)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:22.923194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:23.179641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3f9e581f0a4d64be395f54a5432e7d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:23.219185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3f9e581f0a4d64be395f54a5432e7d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:23.315727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3f9e581f0a4d64be395f54a5432e7d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:23.316432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3f9e581f0a4d64be395f54a5432e7d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:23.317208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3f9e581f0a4d64be395f54a5432e7d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:23.354411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:23.354938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 831/1644] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:22 2022] DELETE /identity/v3/users/6e3f9e581f0a4d64be395f54a5432e7d => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:23.355446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:23.355884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:23.417494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:23.428578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:23.598496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 814/1645] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:23.623686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:23.625081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:23.766420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16ce45-11f5-41d5-a4aa-dbd4fce5c607 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:23.771621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb77a2-605f-47b1-a4a8-a80b619be9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:23.773219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb77a2-605f-47b1-a4a8-a80b619be9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:23.773649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb77a2-605f-47b1-a4a8-a80b619be9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:23.774118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb77a2-605f-47b1-a4a8-a80b619be9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:23.786197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb77a2-605f-47b1-a4a8-a80b619be9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:23.790259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb77a2-605f-47b1-a4a8-a80b619be9a5 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:23.805565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 832/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:24.792710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:24.794646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:24.817061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fccbb-9d9d-49bb-94a0-84dfb20c49c0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:24.832455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:24.833242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:24.833592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:24.833962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] PATH_INFO: `/v3/users/5fcb4c0e6c60442da43fa46daf41345c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:24.837668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fcb4c0e6c60442da43fa46daf41345c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:24.845027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:39:24.845027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Jul 27 08:39:24.845027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: Jul 27 08:39:24.845027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:25.222754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fcb4c0e6c60442da43fa46daf41345c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:25.276000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fcb4c0e6c60442da43fa46daf41345c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:25.378679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fcb4c0e6c60442da43fa46daf41345c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:25.402709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fcb4c0e6c60442da43fa46daf41345c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:25.403483 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fcb4c0e6c60442da43fa46daf41345c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:25.422957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 815/1647] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:24 2022] DELETE /identity/v3/users/5fcb4c0e6c60442da43fa46daf41345c => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:25.425896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:25.427039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:25.427319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:25.473140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:25.474436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:25.593807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 833/1648] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:25.649950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:25.655652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:25.673410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:25.675997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:25.707574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17aaa2-3d4d-4a5a-a4ee-9f989d371b60 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:25.722636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:25.723889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:25.725479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:25.726627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] PATH_INFO: `/v3/projects/d3cea0410a2f4fe88f5f300db0cabbe1` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:25.731619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cea0410a2f4fe88f5f300db0cabbe1)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:25.733230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: WARNING py.warnings [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 27 08:39:25.733230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: warnings.warn(msg) Jul 27 08:39:25.733230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: Jul 27 08:39:25.738641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:25.774921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ac84-c777-43b0-97b3-272dc2b21ff1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:25.776004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:25.779749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:25.780631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:25.781083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:25.781865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:25.794048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:25.796390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:25.810708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 816/1649] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:25.943098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cea0410a2f4fe88f5f300db0cabbe1', 'request_id': 'req-d2493c6b-e370-4b6e-94b7-3eca05a94434'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:25.957453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cea0410a2f4fe88f5f300db0cabbe1', 'request_id': 'req-d2493c6b-e370-4b6e-94b7-3eca05a94434'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.071387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cea0410a2f4fe88f5f300db0cabbe1', 'request_id': 'req-d2493c6b-e370-4b6e-94b7-3eca05a94434'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.108395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] The token cache is being invalidate because project d3cea0410a2f4fe88f5f300db0cabbe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:26.108691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.110558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.112574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 834/1650] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:25 2022] DELETE /identity/v3/projects/d3cea0410a2f4fe88f5f300db0cabbe1 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:39:26.118389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:26.143445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6c669-86c6-4a20-b5e9-ce9c50f4b640 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:26.181977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:26.282307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:26.282991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:26.286636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:26.286636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] PATH_INFO: `/v3/users/f3d1ddd8c35047f8ab09afe70c6819c9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:26.286636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d1ddd8c35047f8ab09afe70c6819c9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:26.287060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:26.400324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:26.401757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:26.469378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2493c6b-e370-4b6e-94b7-3eca05a94434 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:26.479510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:26.480187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:26.483130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:26.484836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] PATH_INFO: `/v3/users/84ad6c639c42426381dc972ea8527f13` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:26.492121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ad6c639c42426381dc972ea8527f13)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:26.499978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:26.603214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d1ddd8c35047f8ab09afe70c6819c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.628035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d1ddd8c35047f8ab09afe70c6819c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.735566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d1ddd8c35047f8ab09afe70c6819c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.754512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d1ddd8c35047f8ab09afe70c6819c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.755876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88a56d81-16c8-4a04-bcea-aa0f1af94b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d1ddd8c35047f8ab09afe70c6819c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.794934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 817/1651] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:26 2022] DELETE /identity/v3/users/f3d1ddd8c35047f8ab09afe70c6819c9 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:26.799593 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ad6c639c42426381dc972ea8527f13'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.820611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ad6c639c42426381dc972ea8527f13'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.910703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ad6c639c42426381dc972ea8527f13'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.911384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ad6c639c42426381dc972ea8527f13'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.912854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ad6c639c42426381dc972ea8527f13'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:26.947637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 835/1652] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:26 2022] DELETE /identity/v3/users/84ad6c639c42426381dc972ea8527f13 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:26.952257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:26.952549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:26.952836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:26.992198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:26.993471 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:27.153750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 818/1653] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:27.173193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:27.174735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:27.306735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f30a2a-27fd-401e-a4f3-c878b7300bf5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:27.311572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:27.312434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:27.313571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:27.315214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:27.338254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:27.340460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:27.369371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 836/1654] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:28.110839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:28.114201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:28.143497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6e6a8-72a2-42c0-a9a7-04f8762aa963 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:28.158867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:28.159836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:28.160254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:28.161566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] PATH_INFO: `/v3/projects/4bf6a3cd5d3545118bf86578de657005` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:28.166126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf6a3cd5d3545118bf86578de657005)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:28.171602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:28.454981 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:28.486730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:28.488265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:28.499852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf6a3cd5d3545118bf86578de657005', 'request_id': 'req-66f7443d-625b-4cc9-bf1c-455c968ae268'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:28.512057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf6a3cd5d3545118bf86578de657005', 'request_id': 'req-66f7443d-625b-4cc9-bf1c-455c968ae268'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:28.514776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dc4a8-0e97-436b-b1f2-01a25b328311 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:28.540146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf6a3cd5d3545118bf86578de657005', 'request_id': 'req-66f7443d-625b-4cc9-bf1c-455c968ae268'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:28.547522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:28.644168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:28.645008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:28.645659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:28.646159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] PATH_INFO: `/v3/projects/173b06cf43cd421eb02bfdfcffcc6a01` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:28.649035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=173b06cf43cd421eb02bfdfcffcc6a01)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:28.651161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:28.660312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] The token cache is being invalidate because project 4bf6a3cd5d3545118bf86578de657005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:28.660838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:28.661773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-66f7443d-625b-4cc9-bf1c-455c968ae268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:28.666528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 819/1655] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:28 2022] DELETE /identity/v3/projects/4bf6a3cd5d3545118bf86578de657005 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:28.671392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dbac3-c7c8-4b4a-82dc-814ed4e61741 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:28.671797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dbac3-c7c8-4b4a-82dc-814ed4e61741 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:28.672178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dbac3-c7c8-4b4a-82dc-814ed4e61741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:28.784493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-dd3dbac3-c7c8-4b4a-82dc-814ed4e61741 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:28.785323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-dd3dbac3-c7c8-4b4a-82dc-814ed4e61741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:28.838549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 820/1656] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:28.847282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ceb12-0980-4b06-b724-fdb7f6292fae None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:28.848168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ceb12-0980-4b06-b724-fdb7f6292fae None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:28.848655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ceb12-0980-4b06-b724-fdb7f6292fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:28.934360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8e2ceb12-0980-4b06-b724-fdb7f6292fae None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:28.936955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8e2ceb12-0980-4b06-b724-fdb7f6292fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:28.956158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173b06cf43cd421eb02bfdfcffcc6a01', 'request_id': 'req-d1f41201-3e8f-4c64-9ae5-72c6944d4486'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:28.991499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173b06cf43cd421eb02bfdfcffcc6a01', 'request_id': 'req-d1f41201-3e8f-4c64-9ae5-72c6944d4486'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.088874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 821/1657] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:29.090738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ceb12-0980-4b06-b724-fdb7f6292fae None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:29.115246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173b06cf43cd421eb02bfdfcffcc6a01', 'request_id': 'req-d1f41201-3e8f-4c64-9ae5-72c6944d4486'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.125174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ceb12-0980-4b06-b724-fdb7f6292fae None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:29.157491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:29.158467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.158905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.159350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.160065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:29.160376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:29.161993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:29.181034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] The token cache is being invalidate because project 173b06cf43cd421eb02bfdfcffcc6a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:29.186810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 822/1658] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:29.188016 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.191719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:29.192296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d1f41201-3e8f-4c64-9ae5-72c6944d4486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.196533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 837/1659] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:28 2022] DELETE /identity/v3/projects/173b06cf43cd421eb02bfdfcffcc6a01 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:39:29.203259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f53d0-6ca6-4a64-b50f-a3168a1393e6 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.209765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f53d0-6ca6-4a64-b50f-a3168a1393e6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.209765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f53d0-6ca6-4a64-b50f-a3168a1393e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.210036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d9349-03f1-48c1-b62b-d81e2eeaa1e8 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:29.215751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:29.216648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.217036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.217499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.218429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:29.221014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:29.270098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b07f53d0-6ca6-4a64-b50f-a3168a1393e6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:29.271383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b07f53d0-6ca6-4a64-b50f-a3168a1393e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:29.282420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 838/1660] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:29.290631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.291135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.291519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.343031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 823/1661] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:29 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:39:29.347280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:29.350990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:29.363990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:29.365415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:29.479978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0c205-23e2-497b-98f6-2d2897094afc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:29.509497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:29.510456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.510883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.511300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.512123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:29.514285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:29.583863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 839/1662] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:29.586283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:29.587954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:29.616710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4385d17-dc8a-4543-8049-532099dfd481 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:29.624710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:29.625620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.626032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.626611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] PATH_INFO: `/v3/projects/3d331b98cf004966a50f39a5fd75a856` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.628411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d331b98cf004966a50f39a5fd75a856)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:29.630591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:29.641838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 824/1663] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:29 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:29.642879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:29.668860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fc426-14c9-48ee-8020-24ded0e51405 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:29.676527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db71a0-b425-4b16-b564-8fbef0f770aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:29.678433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db71a0-b425-4b16-b564-8fbef0f770aa admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.680252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db71a0-b425-4b16-b564-8fbef0f770aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.680774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db71a0-b425-4b16-b564-8fbef0f770aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.681728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db71a0-b425-4b16-b564-8fbef0f770aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:29.682230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db71a0-b425-4b16-b564-8fbef0f770aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:29.684107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db71a0-b425-4b16-b564-8fbef0f770aa admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:29.720917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 825/1664] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:29.722888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1354a-3cd2-4483-a670-7db420a6a9c6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:29.723482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1354a-3cd2-4483-a670-7db420a6a9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:29.724067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1354a-3cd2-4483-a670-7db420a6a9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:29.765881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8ae1354a-3cd2-4483-a670-7db420a6a9c6 None None] MFA Rules not processed for user `26128a80933f4be3bed504d7a2d2785f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:29.768167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8ae1354a-3cd2-4483-a670-7db420a6a9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:29.861720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d331b98cf004966a50f39a5fd75a856', 'request_id': 'req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.888288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d331b98cf004966a50f39a5fd75a856', 'request_id': 'req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.895943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 826/1665] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:29.897690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1354a-3cd2-4483-a670-7db420a6a9c6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:29.926078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1354a-3cd2-4483-a670-7db420a6a9c6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:29.955845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d331b98cf004966a50f39a5fd75a856', 'request_id': 'req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.984126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] The token cache is being invalidate because project 3d331b98cf004966a50f39a5fd75a856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:29.984213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.985157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:29.987240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 840/1666] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:29 2022] DELETE /identity/v3/projects/3d331b98cf004966a50f39a5fd75a856 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:29.988182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:29.989910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:30.075394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:30.076454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:30.077249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:30.077737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:30.080518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:30.085790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:30.086835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf2df4-a56e-4362-849b-a7c0e34f96bc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:30.095178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:30.095808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:30.096086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:30.096358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:30.097079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:30.103027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:30.118687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 827/1667] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:30.120049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:30.122893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:30.246275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 841/1668] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:29 2022] POST /identity/v3/projects => generated 363 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:30.249265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:30.252202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810758-75d0-41e8-928e-500b155f5aa4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:30.254937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:30.270971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:30.359996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:30.400178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:30.401021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:30.401421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:30.401809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:30.406231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:30.427137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b62a8-c878-4182-98f5-2a88dff058e1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:30.442012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:30.442952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:30.443459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:30.443887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] PATH_INFO: `/v3/users/a5b0e464b89f40569d9f23879e483835` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:30.445539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b0e464b89f40569d9f23879e483835)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:30.447989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:30.560718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:30.563541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:30.592703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 828/1669] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:30.594543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:30.596610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:30.620923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12acf1f-4b4a-453c-90aa-288320470118 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:30.640827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:30.641644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:30.642039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:30.642458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:30.643277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:30.645378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:30.680347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 829/1670] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:30.681582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:30.694995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba668f2e-aab4-4e5f-998b-a126e03c3978 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:30.712829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:30.714072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:30.714712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:30.718345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:30.719147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:30.723086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:30.726493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b0e464b89f40569d9f23879e483835'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:30.754472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b0e464b89f40569d9f23879e483835'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:30.868044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 830/1671] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:30 2022] POST /identity/v3/users => generated 282 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:30.869231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:30.887876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b0e464b89f40569d9f23879e483835'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:30.891386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b0e464b89f40569d9f23879e483835'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:30.892183 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b0e464b89f40569d9f23879e483835'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:30.915898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220be8a3-6be2-485e-a63e-73c0a303d89d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:30.935039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 842/1672] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:30 2022] DELETE /identity/v3/users/a5b0e464b89f40569d9f23879e483835 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:30.936399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:30.939273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:30.947048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca46e3-851d-4dc5-afda-c9398084a28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:30.947746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca46e3-851d-4dc5-afda-c9398084a28b admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:30.948033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca46e3-851d-4dc5-afda-c9398084a28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:30.948312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca46e3-851d-4dc5-afda-c9398084a28b admin admin] PATH_INFO: `/v3/projects/7bd4efeeede94f2096d4232446af434e/users/eef1ed0133534e82b6de9fec8ed46da1/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.010440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca46e3-851d-4dc5-afda-c9398084a28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd4efeeede94f2096d4232446af434e, user_id=eef1ed0133534e82b6de9fec8ed46da1, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:31.022002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca46e3-851d-4dc5-afda-c9398084a28b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:31.079590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fef5b-17a1-4e34-85f5-5f17cb6ed116 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:31.096268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:31.096950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.097279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.097561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.098286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:31.100000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:31.120812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 843/1673] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:31.124814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:31.132613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 831/1674] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:30 2022] PUT /identity/v3/projects/7bd4efeeede94f2096d4232446af434e/users/eef1ed0133534e82b6de9fec8ed46da1/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:31.144941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.145265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.145615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.170642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4a658-1105-486c-9ccd-7ce3b17892c9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:31.214004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] MFA Rules not processed for user `eef1ed0133534e82b6de9fec8ed46da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:31.217828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:31.375880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:31.381366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.381782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.382339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.385259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:31.385672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:31.405248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 844/1675] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:31.407192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:31.423604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 832/1676] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:31.429893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:31.430911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b118ad-8d54-4d8e-88fd-66f49ff510b7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:31.431322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:31.438019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87fb2e-edec-48b4-af6e-82947352776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:31.438913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87fb2e-edec-48b4-af6e-82947352776b admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.439304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87fb2e-edec-48b4-af6e-82947352776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.439691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87fb2e-edec-48b4-af6e-82947352776b admin admin] PATH_INFO: `/v3/projects/b5b968d638dd41dcb46ba4c08102694d/users/215991349c634da4ad7ebbfa1f7a09bb/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.467727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87fb2e-edec-48b4-af6e-82947352776b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b968d638dd41dcb46ba4c08102694d, user_id=215991349c634da4ad7ebbfa1f7a09bb, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:31.471964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87fb2e-edec-48b4-af6e-82947352776b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:31.485048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5835da-77fc-46e1-a497-c67347d86bfd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:31.507847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:31.508449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.508923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.509203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.509889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:31.511935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:31.616569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 845/1677] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:31 2022] PUT /identity/v3/projects/b5b968d638dd41dcb46ba4c08102694d/users/215991349c634da4ad7ebbfa1f7a09bb/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:31.620414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca6207-c08f-4ec5-b387-dce4fe214d11 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.621372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca6207-c08f-4ec5-b387-dce4fe214d11 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.624696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca6207-c08f-4ec5-b387-dce4fe214d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.665147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 833/1678] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:31 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:31.668087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:31.698857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f6ca6207-c08f-4ec5-b387-dce4fe214d11 None None] MFA Rules not processed for user `215991349c634da4ad7ebbfa1f7a09bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:31.701577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f6ca6207-c08f-4ec5-b387-dce4fe214d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:31.810746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce916dfb-a199-4139-b544-ffeba7211ae4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:31.821984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:31.825372 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.825680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.825962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.826679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:31.839733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:31.863843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 846/1679] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:31.872533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca6207-c08f-4ec5-b387-dce4fe214d11 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:31.873103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:31.907699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca6207-c08f-4ec5-b387-dce4fe214d11 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:31.915932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:31.916829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:31.917336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:31.917780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:31.919224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:31.929524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:31.958277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:31.969679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 834/1680] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:31 2022] POST /identity/v3/users => generated 299 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:39:31.975029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:31.991449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71aa2a-ad80-4b52-8437-9262de96d653 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:32.003161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:32.004032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.004710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.005129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.009288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:32.014401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:32.043858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 835/1681] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:32.047418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:32.066932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cfbc-f4a3-4ad8-b3ab-54a70b3f895e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:32.067989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 847/1682] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:31 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:32.073436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:32.223846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea005d6-8946-4d57-a17b-bda94e51913c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:32.225102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea005d6-8946-4d57-a17b-bda94e51913c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.225754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea005d6-8946-4d57-a17b-bda94e51913c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.229491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea005d6-8946-4d57-a17b-bda94e51913c admin admin] PATH_INFO: `/v3/projects/4962d678dbd344839d2f57e1951e71f3/users/8df8804293504b42b2954748788d9d05/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.250604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e85064-7239-4b39-8a66-bfc2868184db admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:32.259444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea005d6-8946-4d57-a17b-bda94e51913c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4962d678dbd344839d2f57e1951e71f3, user_id=8df8804293504b42b2954748788d9d05, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:32.259642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322402d5-ae40-4f1d-8546-e01f606e5c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:32.260977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322402d5-ae40-4f1d-8546-e01f606e5c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.261400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322402d5-ae40-4f1d-8546-e01f606e5c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.261860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322402d5-ae40-4f1d-8546-e01f606e5c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.262835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea005d6-8946-4d57-a17b-bda94e51913c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:32.263039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322402d5-ae40-4f1d-8546-e01f606e5c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:32.265651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322402d5-ae40-4f1d-8546-e01f606e5c6b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:32.352468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 836/1683] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:32 2022] PUT /identity/v3/projects/4962d678dbd344839d2f57e1951e71f3/users/8df8804293504b42b2954748788d9d05/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:32.354177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054150d-a275-4c6b-91a5-33eec4f70f75 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.354650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054150d-a275-4c6b-91a5-33eec4f70f75 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.355723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054150d-a275-4c6b-91a5-33eec4f70f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.386499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 848/1684] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:32 2022] POST /identity/v3/users => generated 288 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:32.387844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8751e-c9a0-42b2-9efb-aa1be972305d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.388447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8751e-c9a0-42b2-9efb-aa1be972305d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.388790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8751e-c9a0-42b2-9efb-aa1be972305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.402447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d054150d-a275-4c6b-91a5-33eec4f70f75 None None] MFA Rules not processed for user `37554602ef6d461b97660ad6b337ed98`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:32.404082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d054150d-a275-4c6b-91a5-33eec4f70f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:32.520836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-66d8751e-c9a0-42b2-9efb-aa1be972305d None None] MFA Rules not processed for user `c29a83c5d2ce48eda587f2cec0e959ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:32.525423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-66d8751e-c9a0-42b2-9efb-aa1be972305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:32.566418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 837/1685] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:32.568002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054150d-a275-4c6b-91a5-33eec4f70f75 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:32.597432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:32.726266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054150d-a275-4c6b-91a5-33eec4f70f75 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:32.730991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 849/1686] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:32.731999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8751e-c9a0-42b2-9efb-aa1be972305d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:32.741624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ab9c-c23c-4450-bca3-938308bc6b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:32.744659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ab9c-c23c-4450-bca3-938308bc6b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.746577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ab9c-c23c-4450-bca3-938308bc6b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.746577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ab9c-c23c-4450-bca3-938308bc6b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.749233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ab9c-c23c-4450-bca3-938308bc6b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:32.750974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ab9c-c23c-4450-bca3-938308bc6b2b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:32.767595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8751e-c9a0-42b2-9efb-aa1be972305d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:32.777442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:32.778125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 838/1687] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:32.778339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.778789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.779163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.780158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:32.780376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.780672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.780945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.782437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:32.813797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 850/1688] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:32.813797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:32.816585 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:32.835489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae26f1-e1f5-48d5-a62c-48965ef35fcf admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:32.855295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:32.856216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:32.856586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:32.857067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] PATH_INFO: `/v3/projects/88b260b655614415a0cf3a586035a717` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:32.858920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b260b655614415a0cf3a586035a717)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:32.861117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:32.874937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] MFA Rules not processed for user `37554602ef6d461b97660ad6b337ed98`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:32.876182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:32.901822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 839/1689] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:32.901822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:32.902966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:33.039905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b260b655614415a0cf3a586035a717', 'request_id': 'req-ddb1fc18-76bf-4fb8-96a6-864e141262e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:33.053400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b260b655614415a0cf3a586035a717', 'request_id': 'req-ddb1fc18-76bf-4fb8-96a6-864e141262e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:33.078416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cdae6-4339-41f7-8c34-6fdf7bd09fed None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:33.133408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b260b655614415a0cf3a586035a717', 'request_id': 'req-ddb1fc18-76bf-4fb8-96a6-864e141262e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:33.156336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] The token cache is being invalidate because project 88b260b655614415a0cf3a586035a717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:33.160135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:33.162404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:33.169138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 851/1690] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:32 2022] DELETE /identity/v3/projects/88b260b655614415a0cf3a586035a717 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:33.170449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:33.185797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:33.248433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:33.249743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:33.250992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:33.257301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:33.260115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:33.366383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1fc18-76bf-4fb8-96a6-864e141262e7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:33.373116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:33.374576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:33.375520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:33.375520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] PATH_INFO: `/v3/projects/4962d678dbd344839d2f57e1951e71f3/users/8df8804293504b42b2954748788d9d05/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:33.378991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4962d678dbd344839d2f57e1951e71f3, user_id=8df8804293504b42b2954748788d9d05, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:33.382524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:33.409686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 852/1691] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:33 2022] PUT /identity/v3/projects/4962d678dbd344839d2f57e1951e71f3/users/8df8804293504b42b2954748788d9d05/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:33.412298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:33.412819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:33.414995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:33.421568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:33.428466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 840/1692] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:33.429462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:33.431239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:33.455989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9df99-6fb0-4b01-87e9-5cb5b09b8b23 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:33.589420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6417363-57ad-4780-8f04-ed31c6f911a3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:33.608286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2981daa-92b3-42e2-af88-e1d1557713f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:33.609589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981daa-92b3-42e2-af88-e1d1557713f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:33.613748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981daa-92b3-42e2-af88-e1d1557713f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:33.614396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981daa-92b3-42e2-af88-e1d1557713f8 admin admin] PATH_INFO: `/v3/projects/3ab2e5d462de4cb49eb5c933985230bc/users/103841d29b0948baa77da0f35a492f97/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:33.663279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2981daa-92b3-42e2-af88-e1d1557713f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab2e5d462de4cb49eb5c933985230bc, user_id=103841d29b0948baa77da0f35a492f97, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:33.667104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2981daa-92b3-42e2-af88-e1d1557713f8 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:33.720428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579318b-51a6-4beb-b422-8dc75d23789d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:33.721019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579318b-51a6-4beb-b422-8dc75d23789d service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:33.721303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579318b-51a6-4beb-b422-8dc75d23789d service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:33.721572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579318b-51a6-4beb-b422-8dc75d23789d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:33.727706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3579318b-51a6-4beb-b422-8dc75d23789d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:33.791573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 853/1693] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:33 2022] PUT /identity/v3/projects/3ab2e5d462de4cb49eb5c933985230bc/users/103841d29b0948baa77da0f35a492f97/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:33.793149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9392c-6c1c-4575-af4b-a51a5dbaf316 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:33.795137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9392c-6c1c-4575-af4b-a51a5dbaf316 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:33.795137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9392c-6c1c-4575-af4b-a51a5dbaf316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:33.820998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579318b-51a6-4beb-b422-8dc75d23789d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:33.827215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579318b-51a6-4beb-b422-8dc75d23789d service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:33.843816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 841/1694] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:33.847295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5734522-932d-4b05-b378-270e73b41b87 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:33.850388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5734522-932d-4b05-b378-270e73b41b87 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:33.850388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5734522-932d-4b05-b378-270e73b41b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:33.850727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-00a9392c-6c1c-4575-af4b-a51a5dbaf316 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:33.852808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-00a9392c-6c1c-4575-af4b-a51a5dbaf316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:33.939035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c5734522-932d-4b05-b378-270e73b41b87 None None] MFA Rules not processed for user `8df8804293504b42b2954748788d9d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:33.940364 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c5734522-932d-4b05-b378-270e73b41b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:34.035356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 854/1695] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:34.044454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9392c-6c1c-4575-af4b-a51a5dbaf316 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:34.064251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9392c-6c1c-4575-af4b-a51a5dbaf316 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:34.098500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:34.116806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a49cf4-acfa-4425-9757-26ea4b08218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:34.117612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49cf4-acfa-4425-9757-26ea4b08218e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:34.118194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49cf4-acfa-4425-9757-26ea4b08218e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:34.120595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49cf4-acfa-4425-9757-26ea4b08218e admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:34.121492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a49cf4-acfa-4425-9757-26ea4b08218e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:34.124226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a49cf4-acfa-4425-9757-26ea4b08218e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:34.150592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:34.159940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 855/1696] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:34.163261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:34.163261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:34.163261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:34.249803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:34.253848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:34.258665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 842/1697] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:34.259583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5734522-932d-4b05-b378-270e73b41b87 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:34.276617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 856/1698] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:34.280592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:34.289871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5734522-932d-4b05-b378-270e73b41b87 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:34.292518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:34.294894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:34.295663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:34.296085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:34.296451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] PATH_INFO: `/v3/projects/3ab2e5d462de4cb49eb5c933985230bc/users/103841d29b0948baa77da0f35a492f97/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:34.301312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab2e5d462de4cb49eb5c933985230bc, user_id=103841d29b0948baa77da0f35a492f97, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:34.312765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:34.327032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 843/1699] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:34 2022] PUT /identity/v3/projects/3ab2e5d462de4cb49eb5c933985230bc/users/103841d29b0948baa77da0f35a492f97/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:34.328366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:34.437075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729689c-e8e9-4e3d-a110-56930e261ca7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:34.456201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1134b6-1bb2-4d30-b094-fdfab2e14e4f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:34.460321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:34.461539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:34.461539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:34.462109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:34.462854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:34.463162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:34.465329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:34.512546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 844/1700] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:34.515721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:34.518475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:34.555498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cf489-c05c-44e8-a3c2-7e80a319a49a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:34.569669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:34.570406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:34.570691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:34.570966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] PATH_INFO: `/v3/users/fdcd322e1e404105a230cbe7528eb0bc` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:34.581613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdcd322e1e404105a230cbe7528eb0bc)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:34.586784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:34.632001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7717f7-ef97-46cc-b51a-975f77b3d02e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:34.632650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7717f7-ef97-46cc-b51a-975f77b3d02e service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:34.632942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7717f7-ef97-46cc-b51a-975f77b3d02e service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:34.633353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7717f7-ef97-46cc-b51a-975f77b3d02e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:34.764393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7717f7-ef97-46cc-b51a-975f77b3d02e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:34.768365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7717f7-ef97-46cc-b51a-975f77b3d02e service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:34.797358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 857/1701] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:34.803042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8988dea-448f-4bd5-929d-b6b374b70658 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:34.803367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8988dea-448f-4bd5-929d-b6b374b70658 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:34.803650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8988dea-448f-4bd5-929d-b6b374b70658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:34.898331 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdcd322e1e404105a230cbe7528eb0bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:34.902764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c8988dea-448f-4bd5-929d-b6b374b70658 None None] MFA Rules not processed for user `103841d29b0948baa77da0f35a492f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:34.904592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c8988dea-448f-4bd5-929d-b6b374b70658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:34.970168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdcd322e1e404105a230cbe7528eb0bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.013898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:35.024124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 858/1702] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:35.025916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:35.026614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:35.026614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:35.065573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdcd322e1e404105a230cbe7528eb0bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.074263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdcd322e1e404105a230cbe7528eb0bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.075226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdcd322e1e404105a230cbe7528eb0bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.088940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 845/1703] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:34 2022] DELETE /identity/v3/users/fdcd322e1e404105a230cbe7528eb0bc => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:35.093562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:35.094507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] MFA Rules not processed for user `c370dd8fa3aa407ca22cce8425fcbc25`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:35.095824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:35.098285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:35.257729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a3b8d-8c44-44cc-829c-401ba485a22c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:35.280400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:35.281226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:35.281565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:35.281850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:35.282653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:35.284690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:35.356740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 859/1704] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:35.356740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:35.365904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:35.403129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2b172-e0d0-4915-b28b-0432c20d0115 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:35.412206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 846/1705] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:35 2022] POST /identity/v3/projects => generated 385 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:35.415376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:35.415484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:35.415567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:35.415567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:35.415567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] PATH_INFO: `/v3/users/eef1ed0133534e82b6de9fec8ed46da1` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:35.415788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:35.416421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef1ed0133534e82b6de9fec8ed46da1)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:35.419211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:35.548200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241ebe8-c090-4b1f-a910-3e820f020ea9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:35.555836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed403aed-27d6-4c8e-a4af-b5b063b5c594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:35.556970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403aed-27d6-4c8e-a4af-b5b063b5c594 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:35.557257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403aed-27d6-4c8e-a4af-b5b063b5c594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:35.557536 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403aed-27d6-4c8e-a4af-b5b063b5c594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:35.559640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ed403aed-27d6-4c8e-a4af-b5b063b5c594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:35.643336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:35.676794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403aed-27d6-4c8e-a4af-b5b063b5c594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:35.680216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed403aed-27d6-4c8e-a4af-b5b063b5c594 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:35.696349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 847/1706] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:35 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:35.698125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:35.698541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:35.698932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:35.723878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:35.734781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:35.736263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:35.749390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef1ed0133534e82b6de9fec8ed46da1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.779957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef1ed0133534e82b6de9fec8ed46da1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.909279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef1ed0133534e82b6de9fec8ed46da1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.912814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef1ed0133534e82b6de9fec8ed46da1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.913530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef1ed0133534e82b6de9fec8ed46da1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:35.930197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 848/1707] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:35.931565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:35.937783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:35.947527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 860/1708] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:35 2022] DELETE /identity/v3/users/eef1ed0133534e82b6de9fec8ed46da1 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:35.948394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:35.951939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:36.116695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e0363-31da-4540-b1e4-e46f2e2068e1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.147343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d94fce-0900-448c-bfb2-799fda76ac11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.149185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d94fce-0900-448c-bfb2-799fda76ac11 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.150060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d94fce-0900-448c-bfb2-799fda76ac11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.150532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d94fce-0900-448c-bfb2-799fda76ac11 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.152619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d94fce-0900-448c-bfb2-799fda76ac11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.157758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d94fce-0900-448c-bfb2-799fda76ac11 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.179583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a36e8-5077-42b8-9ce8-587eb0156dfd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.303992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.304542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.304994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.305275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.307718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:36.324071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 861/1709] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:35 2022] POST /identity/v3/users => generated 293 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:36.328489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87a686-d653-4d44-a648-f6301532652c None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.328785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87a686-d653-4d44-a648-f6301532652c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.329083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87a686-d653-4d44-a648-f6301532652c None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.335108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 862/1710] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 27 08:39:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:36.337484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.337773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.338050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.386805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:36.388045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:36.424411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 863/1711] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:36.425314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:36.431546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:36.477630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.478614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.497587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 849/1712] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:36.498864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:36.510922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:36.544402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3620f-502b-4ab8-bf98-bf4bdee35bcd service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.578575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1b99-3f21-4822-ac1e-c69b689affd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.579871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1b99-3f21-4822-ac1e-c69b689affd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.580317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1b99-3f21-4822-ac1e-c69b689affd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.582747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1b99-3f21-4822-ac1e-c69b689affd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.585066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e1b99-3f21-4822-ac1e-c69b689affd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.593937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e1b99-3f21-4822-ac1e-c69b689affd2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.611687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 850/1713] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:36.613202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbd074-166d-49cc-b443-44a718f8ce11 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.613620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbd074-166d-49cc-b443-44a718f8ce11 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.614025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbd074-166d-49cc-b443-44a718f8ce11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.622495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2d86c-6ff9-43c2-bd11-b201167787bb None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.631347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.632015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.632508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.632794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.643640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.646620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.659335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 864/1714] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:36.660362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:36.662636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:36.676706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-88cbd074-166d-49cc-b443-44a718f8ce11 None None] MFA Rules not processed for user `d5c308c33c674f1ea7111c0297174c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:36.681490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-88cbd074-166d-49cc-b443-44a718f8ce11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:36.681995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3c4c6-5e2b-479d-9276-55cb59ec4c67 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.686630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.687479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.688138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.688521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.700361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.701053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 851/1715] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:39:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:36.702647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.703814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbd074-166d-49cc-b443-44a718f8ce11 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:36.716806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 865/1716] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:36.729068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbd074-166d-49cc-b443-44a718f8ce11 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.743790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.743790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.743790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.743790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.757816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.757816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.771669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:36.773898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 852/1717] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:36.777713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:36.784274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01141f83-c8c0-4e67-b71d-f710f05b792d service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.794884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8ec94-67ac-4f9f-9f76-e5c89cc1891c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.795652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8ec94-67ac-4f9f-9f76-e5c89cc1891c service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.795769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2ee44-9526-4876-acb6-e453bc0e8f80 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:36.796343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8ec94-67ac-4f9f-9f76-e5c89cc1891c service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.796720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8ec94-67ac-4f9f-9f76-e5c89cc1891c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.810283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0580dcc-2d5e-49d2-b0b9-8ad612742417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:36.810979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580dcc-2d5e-49d2-b0b9-8ad612742417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:36.811261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580dcc-2d5e-49d2-b0b9-8ad612742417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:36.815108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0580dcc-2d5e-49d2-b0b9-8ad612742417 admin admin] PATH_INFO: `/v3/projects/27823e2b2aee4a1a8b86c1c0968a47e8/users/9092764762fc41059bc989c0ea3a4fdf/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:36.818215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8ec94-67ac-4f9f-9f76-e5c89cc1891c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.822228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8ec94-67ac-4f9f-9f76-e5c89cc1891c service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.844067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 866/1718] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:39:36.868516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0580dcc-2d5e-49d2-b0b9-8ad612742417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27823e2b2aee4a1a8b86c1c0968a47e8, user_id=9092764762fc41059bc989c0ea3a4fdf, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:36.879460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0580dcc-2d5e-49d2-b0b9-8ad612742417 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:36.993338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 853/1719] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:36 2022] PUT /identity/v3/projects/27823e2b2aee4a1a8b86c1c0968a47e8/users/9092764762fc41059bc989c0ea3a4fdf/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:39:37.003058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:37.003549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:37.003985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:37.008642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f5d93-245b-4b58-a335-2caec9437834 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:37.009491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f5d93-245b-4b58-a335-2caec9437834 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:37.010623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f5d93-245b-4b58-a335-2caec9437834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:37.069713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3a0f5d93-245b-4b58-a335-2caec9437834 None None] MFA Rules not processed for user `26128a80933f4be3bed504d7a2d2785f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:37.071704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3a0f5d93-245b-4b58-a335-2caec9437834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:37.098662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] MFA Rules not processed for user `9092764762fc41059bc989c0ea3a4fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:37.103262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:37.251289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 867/1720] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:37.252354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:37.257123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 854/1721] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:37.257286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:37.257600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f5d93-245b-4b58-a335-2caec9437834 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:37.269836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:37.416522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1165771-5eac-4aba-9ebf-c38aeb79cfd0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:37.421776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:37.422728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:37.422858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:37.423097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] PATH_INFO: `/v3/users/215991349c634da4ad7ebbfa1f7a09bb` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:37.424321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] RBAC: Authorizing `identity:delete_user(user_id=215991349c634da4ad7ebbfa1f7a09bb)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:37.429118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:37.560507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f5d93-245b-4b58-a335-2caec9437834 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:37.572697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:37.573943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:37.574507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:37.574856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:37.575689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:37.583988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:37.757629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 855/1722] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:37 2022] POST /identity/v3/projects => generated 387 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:37.763181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:37.785823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215991349c634da4ad7ebbfa1f7a09bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:37.832346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215991349c634da4ad7ebbfa1f7a09bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:37.969094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b32b-7c44-469d-a5f7-4be7b0d3ebde admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:37.983904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215991349c634da4ad7ebbfa1f7a09bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:37.984530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215991349c634da4ad7ebbfa1f7a09bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:37.985314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d586ea41-679a-4a1b-8312-28b14fd010de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215991349c634da4ad7ebbfa1f7a09bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:37.990877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:37.991612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:37.991791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:37.994043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:37.994043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:37.999638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:38.015784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 868/1723] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:37 2022] DELETE /identity/v3/users/215991349c634da4ad7ebbfa1f7a09bb => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:38.021588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:38.026254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:38.026656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:38.095994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:38.097233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:38.200781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 856/1724] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:37 2022] POST /identity/v3/users => generated 300 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:38.204719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:38.209359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:38.253970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a80348-e1b7-4db8-b8b0-6a81732299d6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:38.353351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 869/1725] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:38.384097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:38.386565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:38.543601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c2719-6af9-499e-ad76-872858a2eeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:38.555631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c2719-6af9-499e-ad76-872858a2eeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:38.556079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c2719-6af9-499e-ad76-872858a2eeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:38.557108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c2719-6af9-499e-ad76-872858a2eeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:38.563325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c2719-6af9-499e-ad76-872858a2eeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:38.585140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c2719-6af9-499e-ad76-872858a2eeb3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:38.599941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf4030-e115-44ef-b3ef-96fd9fc7d25f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:38.600434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 857/1726] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:38.601957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75291a2-6699-4d72-bc27-9c1d3a8b8743 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:38.602451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75291a2-6699-4d72-bc27-9c1d3a8b8743 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:38.602871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75291a2-6699-4d72-bc27-9c1d3a8b8743 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:38.604787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 858/1727] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 27 08:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:38.605685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75291a2-6699-4d72-bc27-9c1d3a8b8743 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:38.617235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fd86e-b8d2-48dc-b3cf-9ead0193de5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:38.617878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fd86e-b8d2-48dc-b3cf-9ead0193de5c service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:38.618185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fd86e-b8d2-48dc-b3cf-9ead0193de5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:38.618527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fd86e-b8d2-48dc-b3cf-9ead0193de5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:38.644078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:38.648754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75291a2-6699-4d72-bc27-9c1d3a8b8743 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:38.653862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fd86e-b8d2-48dc-b3cf-9ead0193de5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:38.662905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fd86e-b8d2-48dc-b3cf-9ead0193de5c service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:38.662990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:38.663672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:38.663953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:38.664231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] PATH_INFO: `/v3/projects/b6ff62036009460181e3e6929d9554af/users/832706af60324445b14d767855107b4b/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:38.718609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 870/1728] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:38.720097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ff62036009460181e3e6929d9554af, user_id=832706af60324445b14d767855107b4b, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:38.720250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6c2b-fe4e-4a07-91e6-9338862d0f5d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:38.720439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6c2b-fe4e-4a07-91e6-9338862d0f5d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:38.720716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6c2b-fe4e-4a07-91e6-9338862d0f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:38.722823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:38.773207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e19a6c2b-fe4e-4a07-91e6-9338862d0f5d None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:38.774573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e19a6c2b-fe4e-4a07-91e6-9338862d0f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:38.849694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 859/1729] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:38 2022] PUT /identity/v3/projects/b6ff62036009460181e3e6929d9554af/users/832706af60324445b14d767855107b4b/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:38.853163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:38.856156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:38.938788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 871/1730] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 27 08:39:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:38.941062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6c2b-fe4e-4a07-91e6-9338862d0f5d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:38.986004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6c2b-fe4e-4a07-91e6-9338862d0f5d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:39.091256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abea968-bf7d-4acc-8baf-05880bdf9977 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:39.104202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:39.105763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.107882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.108348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] PATH_INFO: `/v3/projects/47dc670d30af4db294d3b0a6922e3d8e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.112550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47dc670d30af4db294d3b0a6922e3d8e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:39.118026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:39.131702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:39.133667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.135162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.135510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.137007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:39.141472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:39.168902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 872/1731] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:39.169873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:39.267473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c21dd-0125-455a-b8a9-2c351d944842 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:39.274291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:39.275022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.275382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.275661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.277462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:39.289985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:39.291715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:39.302390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 873/1732] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:39.303565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:39.314226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66903aa-774b-4d1a-99ab-c6ab56b10163 service placement] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:39.319758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511c2ba-0798-486f-b152-22b4b2977730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:39.320507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511c2ba-0798-486f-b152-22b4b2977730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.320796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511c2ba-0798-486f-b152-22b4b2977730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.321074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511c2ba-0798-486f-b152-22b4b2977730 admin admin] PATH_INFO: `/v3/projects/b6ff62036009460181e3e6929d9554af/users/832706af60324445b14d767855107b4b/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.323156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511c2ba-0798-486f-b152-22b4b2977730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ff62036009460181e3e6929d9554af, user_id=832706af60324445b14d767855107b4b, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:39.325903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511c2ba-0798-486f-b152-22b4b2977730 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:39.338829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 874/1733] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:39 2022] PUT /identity/v3/projects/b6ff62036009460181e3e6929d9554af/users/832706af60324445b14d767855107b4b/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:39.343579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc1e24-a675-4aeb-8e1d-05ad700c5bd6 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.344054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc1e24-a675-4aeb-8e1d-05ad700c5bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.344484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc1e24-a675-4aeb-8e1d-05ad700c5bd6 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.346347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 875/1734] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 27 08:39:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:39.352389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199153-4f43-4a58-a32a-234824f1652e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.352929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199153-4f43-4a58-a32a-234824f1652e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.353534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199153-4f43-4a58-a32a-234824f1652e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.386432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c3199153-4f43-4a58-a32a-234824f1652e None None] MFA Rules not processed for user `832706af60324445b14d767855107b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:39.387738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c3199153-4f43-4a58-a32a-234824f1652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:39.480026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 876/1735] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:39.480026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.480026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.480026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.500411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] MFA Rules not processed for user `d5c308c33c674f1ea7111c0297174c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:39.501664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:39.571788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 877/1736] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:39:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:39.573244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:39.574661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:39.631698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b10dc-cd1d-4ee1-8169-cab5b5599a99 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:39.635420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:39.636042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.636325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.636599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.646541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:39.648017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:39.657884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 878/1737] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:39.658979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:39.660430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:39.673890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ceee3f-d065-440d-97cd-6e4b45c1e767 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:39.677757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:39.678585 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:39.679085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:39.679471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:39.687132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:39.688571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:39.697156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 879/1738] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:40.850273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:40.855023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:40.876121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47dc670d30af4db294d3b0a6922e3d8e', 'request_id': 'req-a672b33d-9768-40f1-8e1a-38d48bd54e4a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:40.897321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47dc670d30af4db294d3b0a6922e3d8e', 'request_id': 'req-a672b33d-9768-40f1-8e1a-38d48bd54e4a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:40.911099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47dc670d30af4db294d3b0a6922e3d8e', 'request_id': 'req-a672b33d-9768-40f1-8e1a-38d48bd54e4a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.027063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b309e59-8cbc-4a5a-8b6e-3c8f9813784a service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:41.056485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] The token cache is being invalidate because project 47dc670d30af4db294d3b0a6922e3d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:41.056485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.056485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a672b33d-9768-40f1-8e1a-38d48bd54e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.056485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 860/1739] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:38 2022] DELETE /identity/v3/projects/47dc670d30af4db294d3b0a6922e3d8e => generated 0 bytes in 2203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:41.058599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f485b7-a16f-4b6d-8889-7ab7c2656751 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:41.059722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f485b7-a16f-4b6d-8889-7ab7c2656751 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:41.060200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f485b7-a16f-4b6d-8889-7ab7c2656751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:41.161264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d8f485b7-a16f-4b6d-8889-7ab7c2656751 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:41.168730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d8f485b7-a16f-4b6d-8889-7ab7c2656751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:41.247103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:41.252156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:41.253133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:41.253435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] PATH_INFO: `/v3/projects/7bd4efeeede94f2096d4232446af434e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:41.254832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd4efeeede94f2096d4232446af434e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:41.258411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:41.374196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 861/1740] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:41.379022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4c5d-8859-4cdc-a4c2-6f0395310fd9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:41.379022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4c5d-8859-4cdc-a4c2-6f0395310fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:41.379022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb4c5d-8859-4cdc-a4c2-6f0395310fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:41.450739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bfeb4c5d-8859-4cdc-a4c2-6f0395310fd9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:41.463495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bfeb4c5d-8859-4cdc-a4c2-6f0395310fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:41.464033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:41.467230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd4efeeede94f2096d4232446af434e', 'request_id': 'req-e749a38d-9fd2-4934-a642-7a1340cb1069'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.498245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd4efeeede94f2096d4232446af434e', 'request_id': 'req-e749a38d-9fd2-4934-a642-7a1340cb1069'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.619543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 862/1741] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:41.623836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb4c5d-8859-4cdc-a4c2-6f0395310fd9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:41.633626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd4efeeede94f2096d4232446af434e', 'request_id': 'req-e749a38d-9fd2-4934-a642-7a1340cb1069'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.641992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb4c5d-8859-4cdc-a4c2-6f0395310fd9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:41.653233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] The token cache is being invalidate because project 7bd4efeeede94f2096d4232446af434e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:41.653556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.654766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:41.655640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:41.656119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:41.656408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:41.656679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:41.658413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 880/1742] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:40 2022] DELETE /identity/v3/projects/7bd4efeeede94f2096d4232446af434e => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:39:41.662715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:41.663043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:41.664359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:41.666372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:41.667072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:41.688988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 863/1743] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:39:41.691221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749a38d-9fd2-4934-a642-7a1340cb1069 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:41.693589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:41.699736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:41.707414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:41.708250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:41.709029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:41.709302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] PATH_INFO: `/v3/users/8df8804293504b42b2954748788d9d05` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:41.715454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df8804293504b42b2954748788d9d05)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:41.717761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:41.728706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012d1a4-e336-4b26-a819-ed147fafcbcd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:41.739174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:41.740297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:41.740843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:41.742091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:41.743061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:41.746225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:41.931457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 864/1744] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:41 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:41.933078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:41.935623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:42.019398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df8804293504b42b2954748788d9d05'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.047991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df8804293504b42b2954748788d9d05'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.093082 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6316b-51d9-4070-8f53-175b329c3183 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:42.098551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:42.100269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:42.100722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:42.101180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] PATH_INFO: `/v3/projects/b5b968d638dd41dcb46ba4c08102694d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:42.103296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b968d638dd41dcb46ba4c08102694d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:42.106745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:42.175934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df8804293504b42b2954748788d9d05'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.182644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df8804293504b42b2954748788d9d05'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.186806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df8804293504b42b2954748788d9d05'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.220560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 881/1745] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:41 2022] DELETE /identity/v3/users/8df8804293504b42b2954748788d9d05 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:42.220560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:42.230369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:42.276846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b968d638dd41dcb46ba4c08102694d', 'request_id': 'req-b0ca9222-d485-4ee0-ae8a-13c612538d3b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.293639 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b968d638dd41dcb46ba4c08102694d', 'request_id': 'req-b0ca9222-d485-4ee0-ae8a-13c612538d3b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.332348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b968d638dd41dcb46ba4c08102694d', 'request_id': 'req-b0ca9222-d485-4ee0-ae8a-13c612538d3b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.382692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36384b34-5e2d-4cdf-b075-930b60e6fb06 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:42.394690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:42.458956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] The token cache is being invalidate because project b5b968d638dd41dcb46ba4c08102694d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:42.464554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.466389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b0ca9222-d485-4ee0-ae8a-13c612538d3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:42.468561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 865/1746] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:41 2022] DELETE /identity/v3/projects/b5b968d638dd41dcb46ba4c08102694d => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:42.475234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:42.475594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:42.475875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:42.487732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:42.541977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] MFA Rules not processed for user `c29a83c5d2ce48eda587f2cec0e959ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:42.543129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:42.608815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:42.609465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:42.609752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:42.610056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:42.618089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:42.623012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:42.750456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 866/1747] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:42.750456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:42.753729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:42.791628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7e863-2e56-4c8a-8e5f-9fac5ccf66cf None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:42.801354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 882/1748] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:42 2022] POST /identity/v3/users => generated 287 bytes in 583 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:42.806126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:42.808985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:42.819467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:42.820144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:42.820426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:42.825521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] PATH_INFO: `/v3/users/103841d29b0948baa77da0f35a492f97` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:42.830265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=103841d29b0948baa77da0f35a492f97)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:42.838377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:42.855358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e616535-22ff-48b6-bc84-9013b26b81ec admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:42.875796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:42.877108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:42.882442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:42.882788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:42.885556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:42.887378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:42.935088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 883/1749] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:42.936105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:42.957148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f77f06-f244-4764-812b-c58d341b29ae admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:42.967039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:42.967692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:42.967975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:42.968260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:42.968964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:42.974432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:42.996169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 884/1750] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:43.014120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:43.038631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83873b-2b75-4fd4-a2c1-0d78d4a7b29b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:43.056413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:43.057398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:43.057825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:43.058341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] PATH_INFO: `/v3/projects/f154fba3c0b846a8aac5d0f1496bf858/users/eac13b5b885f49648f56176349e6c0dc/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:43.087598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f154fba3c0b846a8aac5d0f1496bf858, user_id=eac13b5b885f49648f56176349e6c0dc, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:43.092116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:43.137793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103841d29b0948baa77da0f35a492f97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.157290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103841d29b0948baa77da0f35a492f97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.184271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 885/1751] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:42 2022] PUT /identity/v3/projects/f154fba3c0b846a8aac5d0f1496bf858/users/eac13b5b885f49648f56176349e6c0dc/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:39:43.194838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:43.196313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:43.268909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103841d29b0948baa77da0f35a492f97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.277571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103841d29b0948baa77da0f35a492f97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.280887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103841d29b0948baa77da0f35a492f97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.284245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be2592-7886-4de0-93a5-a1019b3bd3ba admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:43.291956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:43.292795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:43.292932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:43.293211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] PATH_INFO: `/v3/users/9092764762fc41059bc989c0ea3a4fdf` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:43.298740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9092764762fc41059bc989c0ea3a4fdf)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:43.303053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:43.312718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 867/1752] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:42 2022] DELETE /identity/v3/users/103841d29b0948baa77da0f35a492f97 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:43.317258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:43.322437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:43.378701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:43.442066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66d6ba-5e2f-4337-82ea-2f24950304ad admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:43.451993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:43.452853 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:43.453211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:43.453562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] PATH_INFO: `/v3/projects/4962d678dbd344839d2f57e1951e71f3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:43.455087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4962d678dbd344839d2f57e1951e71f3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:43.457867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:43.577191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9092764762fc41059bc989c0ea3a4fdf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.597597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9092764762fc41059bc989c0ea3a4fdf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.664352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4962d678dbd344839d2f57e1951e71f3', 'request_id': 'req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.673118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4962d678dbd344839d2f57e1951e71f3', 'request_id': 'req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.685015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4962d678dbd344839d2f57e1951e71f3', 'request_id': 'req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.700106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9092764762fc41059bc989c0ea3a4fdf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.700963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9092764762fc41059bc989c0ea3a4fdf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.701986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-c16e8184-4bfd-4afc-b155-72386f3c90d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9092764762fc41059bc989c0ea3a4fdf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.730497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 886/1753] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:43 2022] DELETE /identity/v3/users/9092764762fc41059bc989c0ea3a4fdf => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:43.733507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13417827-e68c-4ea4-96ed-80e1b9a5189d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:43.733507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13417827-e68c-4ea4-96ed-80e1b9a5189d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:43.733507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13417827-e68c-4ea4-96ed-80e1b9a5189d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:43.778234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-13417827-e68c-4ea4-96ed-80e1b9a5189d None None] MFA Rules not processed for user `eac13b5b885f49648f56176349e6c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:43.779545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-13417827-e68c-4ea4-96ed-80e1b9a5189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:43.828294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] The token cache is being invalidate because project 4962d678dbd344839d2f57e1951e71f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:43.828294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.828294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ded40112-a429-44f1-85b7-dd4e6f1fc7d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:43.828294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 868/1754] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:43 2022] DELETE /identity/v3/projects/4962d678dbd344839d2f57e1951e71f3 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:43.828294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2268-fb87-44e2-9678-9c753f9c52c1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:43.828672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2268-fb87-44e2-9678-9c753f9c52c1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:43.828672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2268-fb87-44e2-9678-9c753f9c52c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:43.890267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e93d2268-fb87-44e2-9678-9c753f9c52c1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:43.893012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e93d2268-fb87-44e2-9678-9c753f9c52c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:44.011532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 887/1755] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:44.022270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.023165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.023691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.059861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:44.061934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:44.130784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 869/1756] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:44.157208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 888/1757] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:44.168745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329510f-8381-487f-be14-08ba15aa16f6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.169339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329510f-8381-487f-be14-08ba15aa16f6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.170196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329510f-8381-487f-be14-08ba15aa16f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.198863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:44.200515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:44.207369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5329510f-8381-487f-be14-08ba15aa16f6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:44.211307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5329510f-8381-487f-be14-08ba15aa16f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:44.247520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 870/1758] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:44.251380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329510f-8381-487f-be14-08ba15aa16f6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:44.286618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329510f-8381-487f-be14-08ba15aa16f6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:44.300361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:44.300361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.300361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.300361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.302871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:44.303269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:44.306486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:44.327808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 871/1759] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:44.328758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:44.330720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:44.388972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420af30-b8df-4e85-9631-560b9506dfbf None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:44.398423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:44.399042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.399386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.399847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.418736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:44.420214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:44.446170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 889/1760] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:44.449789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:44.503676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d6eb0-20e0-4004-8116-a6a5a805d945 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:44.511629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:44.513639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.514062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.514588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.515557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:44.518457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:44.553656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6a010-1b7a-40bc-bf29-06df6e71a7e3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:44.581627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:44.582903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.583982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.585825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.608606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:44.612839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:44.631875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 872/1761] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:44.633039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:44.634732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:44.654846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa2a03-cea7-4d32-8aad-d9dec72afe44 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:44.670971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:44.671892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.674244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.674895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] PATH_INFO: `/v3/projects/3ab2e5d462de4cb49eb5c933985230bc` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.678219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab2e5d462de4cb49eb5c933985230bc)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:44.680100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 890/1762] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:44 2022] POST /identity/v3/projects => generated 391 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:44.680416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:44.681371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:44.873488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab2e5d462de4cb49eb5c933985230bc', 'request_id': 'req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:44.888415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85044e-a16f-4a84-affc-71e9d5f0dbf7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:44.912575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab2e5d462de4cb49eb5c933985230bc', 'request_id': 'req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:44.921373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab2e5d462de4cb49eb5c933985230bc', 'request_id': 'req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:44.997285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065368b-e8d7-49df-b4b8-73fce56dc3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:44.998010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065368b-e8d7-49df-b4b8-73fce56dc3ab admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:44.998402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065368b-e8d7-49df-b4b8-73fce56dc3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:44.998759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065368b-e8d7-49df-b4b8-73fce56dc3ab admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:44.999503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065368b-e8d7-49df-b4b8-73fce56dc3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:45.001771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065368b-e8d7-49df-b4b8-73fce56dc3ab admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:45.031619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] The token cache is being invalidate because project 3ab2e5d462de4cb49eb5c933985230bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:45.032027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.033303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bd9c3ac5-8f1b-4e0b-ae3e-6b9f1089c036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.039767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 873/1763] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:44 2022] DELETE /identity/v3/projects/3ab2e5d462de4cb49eb5c933985230bc => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:45.042196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.042563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.042914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.068276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] MFA Rules not processed for user `c370dd8fa3aa407ca22cce8425fcbc25`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:45.069928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.138917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 874/1764] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:45.140129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:45.142227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.158591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4f007-c45d-44ef-a3af-5d6098b0ffb9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:45.169896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 891/1765] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:44 2022] POST /identity/v3/users => generated 296 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:45.170660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.171093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.171373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.178094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:45.178902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.179253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.179621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] PATH_INFO: `/v3/users/eac13b5b885f49648f56176349e6c0dc` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.181009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac13b5b885f49648f56176349e6c0dc)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:45.185515 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:45.263687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:45.267676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.275290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:45.291626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 892/1766] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:45.292630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:45.295038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.308919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50e306-3f03-471a-90ea-0c274fb9529f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:45.318535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58260d49-c690-41e1-8df6-6fbb1f1bee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:45.319254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260d49-c690-41e1-8df6-6fbb1f1bee4d admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.319600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260d49-c690-41e1-8df6-6fbb1f1bee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.319965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260d49-c690-41e1-8df6-6fbb1f1bee4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.320748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58260d49-c690-41e1-8df6-6fbb1f1bee4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:45.322558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58260d49-c690-41e1-8df6-6fbb1f1bee4d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:45.344125 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 893/1767] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:45.350480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934552eb-ddc8-4e65-b3a6-637eebd9cce8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.350890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934552eb-ddc8-4e65-b3a6-637eebd9cce8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.351080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934552eb-ddc8-4e65-b3a6-637eebd9cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.360808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:45.414506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-934552eb-ddc8-4e65-b3a6-637eebd9cce8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:45.415864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-934552eb-ddc8-4e65-b3a6-637eebd9cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.430558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 894/1768] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:45.430558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934552eb-ddc8-4e65-b3a6-637eebd9cce8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:45.441297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac13b5b885f49648f56176349e6c0dc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.479320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934552eb-ddc8-4e65-b3a6-637eebd9cce8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:45.491878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:45.493260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac13b5b885f49648f56176349e6c0dc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.494318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.496621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.497058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.498079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:45.502101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:45.539738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 895/1769] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:45.540322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:45.544752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.579312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac13b5b885f49648f56176349e6c0dc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.611099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3716b75-dc56-437d-b3b4-7690adc17af5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:45.613172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac13b5b885f49648f56176349e6c0dc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.616356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac13b5b885f49648f56176349e6c0dc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.620557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:45.621032 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.621564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.621839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] PATH_INFO: `/v3/projects/27823e2b2aee4a1a8b86c1c0968a47e8` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.623127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27823e2b2aee4a1a8b86c1c0968a47e8)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:45.625170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:45.646277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 875/1770] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:45 2022] DELETE /identity/v3/users/eac13b5b885f49648f56176349e6c0dc => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:45.647484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:45.649205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.772667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27823e2b2aee4a1a8b86c1c0968a47e8', 'request_id': 'req-b0a3901b-3edf-4dce-a0bb-c0f551e21900'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.781514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27823e2b2aee4a1a8b86c1c0968a47e8', 'request_id': 'req-b0a3901b-3edf-4dce-a0bb-c0f551e21900'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.786030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9013d-9e41-4b53-a86b-463ce1de5cdd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:45.868186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:45.868186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.868186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.868186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.868612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:45.868612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:45.869798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:45.874734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27823e2b2aee4a1a8b86c1c0968a47e8', 'request_id': 'req-b0a3901b-3edf-4dce-a0bb-c0f551e21900'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.901426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 876/1771] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:45.903351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:45.905199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] The token cache is being invalidate because project 27823e2b2aee4a1a8b86c1c0968a47e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:45.909031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:45.921905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.925022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b0a3901b-3edf-4dce-a0bb-c0f551e21900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:45.931279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 896/1772] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:45 2022] DELETE /identity/v3/projects/27823e2b2aee4a1a8b86c1c0968a47e8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:45.933253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.937393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.937537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e471a8-1458-4983-8185-f0de0fedf786 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:45.938424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.944453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fffeb7-f75c-4fdf-bfa4-0fe69e7a4991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:45.945188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fffeb7-f75c-4fdf-bfa4-0fe69e7a4991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:45.945854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fffeb7-f75c-4fdf-bfa4-0fe69e7a4991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:45.946529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fffeb7-f75c-4fdf-bfa4-0fe69e7a4991 admin admin] PATH_INFO: `/v3/projects/398e5916da014b0683dd7ba1664d49a0/users/8cc50f40c890475f9b3ba03c51b74a72/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:45.969751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fffeb7-f75c-4fdf-bfa4-0fe69e7a4991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=398e5916da014b0683dd7ba1664d49a0, user_id=8cc50f40c890475f9b3ba03c51b74a72, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:45.976665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fffeb7-f75c-4fdf-bfa4-0fe69e7a4991 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:46.039198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:46.040580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:46.086904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 877/1773] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:45 2022] PUT /identity/v3/projects/398e5916da014b0683dd7ba1664d49a0/users/8cc50f40c890475f9b3ba03c51b74a72/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:46.089581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:46.089875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:46.092027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:46.133522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:46.136149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:46.182285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 897/1774] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:39:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:46.184515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:46.186824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:46.246595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6990-89bd-479f-ac53-37a1a1667325 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:46.264776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 878/1775] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:46.275091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:46.280345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:46.324873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf1acf-ff20-4fda-9fd3-824e83903d44 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:46.338308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:46.338978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:46.339263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:46.339541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] PATH_INFO: `/v3/users/832706af60324445b14d767855107b4b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:46.347485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=832706af60324445b14d767855107b4b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:46.349526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:46.439808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fec5ee-d4c8-4fa6-9f37-5ea4dbdf84e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:46.439808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fec5ee-d4c8-4fa6-9f37-5ea4dbdf84e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:46.439808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fec5ee-d4c8-4fa6-9f37-5ea4dbdf84e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:46.439808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fec5ee-d4c8-4fa6-9f37-5ea4dbdf84e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:46.440402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fec5ee-d4c8-4fa6-9f37-5ea4dbdf84e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:46.440402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fec5ee-d4c8-4fa6-9f37-5ea4dbdf84e5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:46.661341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 898/1776] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:46 2022] POST /identity/v3/projects => generated 357 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:46.663169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:46.663597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:46.664066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:46.705958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] MFA Rules not processed for user `8cc50f40c890475f9b3ba03c51b74a72`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:46.709815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:46.726581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '832706af60324445b14d767855107b4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:46.748040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '832706af60324445b14d767855107b4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:46.825907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 899/1777] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:46.827134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:46.828706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:46.832235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '832706af60324445b14d767855107b4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:46.846779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '832706af60324445b14d767855107b4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:46.847783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '832706af60324445b14d767855107b4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:46.867475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 879/1778] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:46 2022] DELETE /identity/v3/users/832706af60324445b14d767855107b4b => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:46.869280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:46.871152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:46.931121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c416b-e16b-4a25-985b-30d2c3a07006 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:46.983970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c560ef-cbf2-480e-b8db-9cea307b2604 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:46.993050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:46.993997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:46.994470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:46.994881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:46.998589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:46.998589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:46.998589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:46.998589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:46.998932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:47.000342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.002359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.016127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.018044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.028632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 900/1779] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:47.029512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:47.031146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:47.135081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba09669-0f98-4314-aea9-dceeb3afa83d service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:47.140669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:47.141815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:47.141815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:47.141815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:47.143772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:47.178522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 880/1780] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:46 2022] POST /identity/v3/users => generated 279 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:47.179851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:47.187356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:47.215353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8be4d-ccb7-4637-ac3b-8b279b9e79dd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:47.220747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.222240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.228672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:47.230128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:47.230678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:47.231164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:47.232076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.233862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 901/1781] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:39:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:39:47.234743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:47.242598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.245886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79a345-7b20-4094-bd38-c27c48a846ab service swift] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:47.251327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:47.252035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:47.252321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:47.252667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:47.253431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.255164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.273869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 902/1782] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:47.278109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:47.298368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e123-115f-40ef-a406-9d45c35eb459 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:47.304796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:47.305812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:47.306346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:47.306834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:47.308169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.310631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.369171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 881/1783] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:47 2022] POST /identity/v3/projects => generated 393 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:47.370441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:47.412576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 903/1784] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:47.412576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:47.468097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ce92c-59a3-4711-84e7-3ef9b2df8f5c admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:47.499012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:47.665814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:47.705673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5ce69-4bcb-4c09-8ef6-de92265b2110 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:47.718054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:47.720259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:47.720638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:47.720827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:47.721938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.726132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e102a33-029d-428e-ad3e-b3692e9df0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:47.728784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e102a33-029d-428e-ad3e-b3692e9df0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:47.729021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e102a33-029d-428e-ad3e-b3692e9df0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:47.729263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e102a33-029d-428e-ad3e-b3692e9df0fe admin admin] PATH_INFO: `/v3/projects/13e24414aaae47afaadde1a97ff9ab76/users/d74fceb9edc945cda61f8a6883ba4f74/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:47.730237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.800208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e102a33-029d-428e-ad3e-b3692e9df0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=13e24414aaae47afaadde1a97ff9ab76, user_id=d74fceb9edc945cda61f8a6883ba4f74, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:47.806448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e102a33-029d-428e-ad3e-b3692e9df0fe admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:47.904285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 904/1785] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:47 2022] PUT /identity/v3/projects/13e24414aaae47afaadde1a97ff9ab76/users/d74fceb9edc945cda61f8a6883ba4f74/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:47.904285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828e34d-f9f4-4b49-b394-e85a718f397c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:47.904285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828e34d-f9f4-4b49-b394-e85a718f397c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:47.904285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828e34d-f9f4-4b49-b394-e85a718f397c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:47.904955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 882/1786] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:47 2022] POST /identity/v3/users => generated 303 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:47.904955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:47.936821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432e36e8-be8a-4a1c-bc57-dd317b9b4004 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:47.989497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4828e34d-f9f4-4b49-b394-e85a718f397c None None] MFA Rules not processed for user `d5c308c33c674f1ea7111c0297174c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:47.997672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4828e34d-f9f4-4b49-b394-e85a718f397c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:48.163900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:48.164802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:48.165262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:48.165687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:48.166645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:48.172243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:48.184329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 905/1787] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Jul 27 08:39:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:48.186265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9214f-02a0-4325-a959-0075f77dd362 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:48.191953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 883/1788] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:48.193994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:48.201363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:48.211718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9214f-02a0-4325-a959-0075f77dd362 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:48.212061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9214f-02a0-4325-a959-0075f77dd362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:48.220165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c9d67-8d03-44ca-ba5f-6646587b9e43 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:48.240580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:48.241689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:48.242195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:48.243188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] PATH_INFO: `/v3/projects/b6ff62036009460181e3e6929d9554af` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:48.244703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ff62036009460181e3e6929d9554af)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:48.247945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:48.361358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-91a9214f-02a0-4325-a959-0075f77dd362 None None] MFA Rules not processed for user `d74fceb9edc945cda61f8a6883ba4f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:48.361358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-91a9214f-02a0-4325-a959-0075f77dd362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:48.476359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ff62036009460181e3e6929d9554af', 'request_id': 'req-11be84e5-2068-4a26-b75b-9ba23539f242'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:48.502974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ff62036009460181e3e6929d9554af', 'request_id': 'req-11be84e5-2068-4a26-b75b-9ba23539f242'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:48.528736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ff62036009460181e3e6929d9554af', 'request_id': 'req-11be84e5-2068-4a26-b75b-9ba23539f242'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:48.563216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 906/1789] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:48 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:48.564540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9214f-02a0-4325-a959-0075f77dd362 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:48.584953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9214f-02a0-4325-a959-0075f77dd362 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:48.669950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] The token cache is being invalidate because project b6ff62036009460181e3e6929d9554af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:48.671394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:48.674389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:48.681468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 884/1790] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:48 2022] DELETE /identity/v3/projects/b6ff62036009460181e3e6929d9554af => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:48.682532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:48.689637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:48.733913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:48.734958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:48.735361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:48.735761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] PATH_INFO: `/v3/projects/ca52792fdd7148589ae321c1309430f7/users/8f5f98eb8fd142e1b278ef50dd0dec8e/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:48.826963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca52792fdd7148589ae321c1309430f7, user_id=8f5f98eb8fd142e1b278ef50dd0dec8e, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:48.826963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:48.857640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be84e5-2068-4a26-b75b-9ba23539f242 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:48.876738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:48.877868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:48.878458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:48.878872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] PATH_INFO: `/v3/projects/f154fba3c0b846a8aac5d0f1496bf858` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:48.900913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f154fba3c0b846a8aac5d0f1496bf858)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:48.900913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:49.070664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 907/1791] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:48 2022] PUT /identity/v3/projects/ca52792fdd7148589ae321c1309430f7/users/8f5f98eb8fd142e1b278ef50dd0dec8e/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:49.071530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:49.073418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.100795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f154fba3c0b846a8aac5d0f1496bf858', 'request_id': 'req-99139ca5-3947-4aab-bbc6-6020601b677e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:49.104700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f154fba3c0b846a8aac5d0f1496bf858', 'request_id': 'req-99139ca5-3947-4aab-bbc6-6020601b677e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:49.116301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f154fba3c0b846a8aac5d0f1496bf858', 'request_id': 'req-99139ca5-3947-4aab-bbc6-6020601b677e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:49.164373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3f05c-dd42-4183-8b9c-5f00b0cbf82f admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:49.248309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:49.250961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:49.259452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.259452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.259452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.259452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.262098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] The token cache is being invalidate because project f154fba3c0b846a8aac5d0f1496bf858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:49.262488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:49.266813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:49.267899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:49.268258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:49.273170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 885/1792] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:48 2022] DELETE /identity/v3/projects/f154fba3c0b846a8aac5d0f1496bf858 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:49.273170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:49.274302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.280338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 908/1793] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:49.281341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:49.282927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.297148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d62768-0fa0-45b0-b785-b8fa7b36f230 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:49.306703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67035b28-2815-43f5-9c4b-f9e88eae7ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:49.307785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67035b28-2815-43f5-9c4b-f9e88eae7ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.308378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67035b28-2815-43f5-9c4b-f9e88eae7ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.309076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67035b28-2815-43f5-9c4b-f9e88eae7ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.311727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-67035b28-2815-43f5-9c4b-f9e88eae7ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.323185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67035b28-2815-43f5-9c4b-f9e88eae7ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:49.325133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67035b28-2815-43f5-9c4b-f9e88eae7ce0 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:49.333065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:49.339262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 909/1794] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:49.340979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.341415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.341831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.372641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:49.374074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.497721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99139ca5-3947-4aab-bbc6-6020601b677e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:49.500754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 910/1795] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:49.502224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:49.503935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:49.504774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.505066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.506251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.506365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.507097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:49.510004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:49.533169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa799a-495e-4d49-a6b2-fab8a968040c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:49.553854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78e280-590c-4702-954f-1d40753609fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:49.553854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e280-590c-4702-954f-1d40753609fe admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.553854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e280-590c-4702-954f-1d40753609fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.553854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e280-590c-4702-954f-1d40753609fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.554845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78e280-590c-4702-954f-1d40753609fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:49.554845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78e280-590c-4702-954f-1d40753609fe admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:49.568078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 911/1796] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:49.571027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fe3ad-f3a5-4a38-a0bf-dc1b95a9f5bc None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.571027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fe3ad-f3a5-4a38-a0bf-dc1b95a9f5bc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.571027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fe3ad-f3a5-4a38-a0bf-dc1b95a9f5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.597801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-0c9fe3ad-f3a5-4a38-a0bf-dc1b95a9f5bc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:49.601865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0c9fe3ad-f3a5-4a38-a0bf-dc1b95a9f5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.622569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 912/1797] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:49.631398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9285a-070f-4b3d-88bb-56e239727855 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.631686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9285a-070f-4b3d-88bb-56e239727855 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.631964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9285a-070f-4b3d-88bb-56e239727855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.643865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 886/1798] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:49 2022] POST /identity/v3/projects => generated 357 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:49.645005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:49.654255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3ab9285a-070f-4b3d-88bb-56e239727855 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:49.655312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3ab9285a-070f-4b3d-88bb-56e239727855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.733319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe6896-88aa-4542-a7c0-022ba86c65db admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:49.745077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 913/1799] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:49.745077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d252-b778-4ea8-bf76-b6fad49013a9 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.745077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d252-b778-4ea8-bf76-b6fad49013a9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.745077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d252-b778-4ea8-bf76-b6fad49013a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.748635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:49.749371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:49.749648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:49.749998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] PATH_INFO: `/v3/projects/ca52792fdd7148589ae321c1309430f7/users/8f5f98eb8fd142e1b278ef50dd0dec8e/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:49.755285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca52792fdd7148589ae321c1309430f7, user_id=8f5f98eb8fd142e1b278ef50dd0dec8e, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:49.766749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:49.771127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4643d252-b778-4ea8-bf76-b6fad49013a9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:49.773913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4643d252-b778-4ea8-bf76-b6fad49013a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:49.808263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 887/1800] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:49 2022] PUT /identity/v3/projects/ca52792fdd7148589ae321c1309430f7/users/8f5f98eb8fd142e1b278ef50dd0dec8e/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:49.811026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:49.812605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:49.993656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 914/1801] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:49.997225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643d252-b778-4ea8-bf76-b6fad49013a9 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.024610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643d252-b778-4ea8-bf76-b6fad49013a9 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:50.026898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36b09-32eb-4995-b018-b4b2ed4d5742 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:50.041404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:50.042297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629bb1c-5699-436e-ba54-eb5846805303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:50.042441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.042991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629bb1c-5699-436e-ba54-eb5846805303 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.043109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.043275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629bb1c-5699-436e-ba54-eb5846805303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.043527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.043629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629bb1c-5699-436e-ba54-eb5846805303 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.044326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629bb1c-5699-436e-ba54-eb5846805303 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:50.044558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:50.044665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629bb1c-5699-436e-ba54-eb5846805303 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:50.049041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:50.052163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629bb1c-5699-436e-ba54-eb5846805303 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:50.071391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 915/1802] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:50.073095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75362702-2398-4768-89af-5a9bbe635707 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.073382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75362702-2398-4768-89af-5a9bbe635707 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.073657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75362702-2398-4768-89af-5a9bbe635707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.136446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-75362702-2398-4768-89af-5a9bbe635707 None None] MFA Rules not processed for user `8f5f98eb8fd142e1b278ef50dd0dec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:50.138377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-75362702-2398-4768-89af-5a9bbe635707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:50.180866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 888/1803] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:49 2022] POST /identity/v3/users => generated 285 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:50.180866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.209506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3b606-9bfd-4625-b5c0-1be1f59e87e8 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:50.209506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:50.209506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.209506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.210086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.210086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:39:50.210086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:50.212408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:50.234402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 889/1804] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:39:50.235743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.258609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e9351-c276-4193-8ea3-60d77b9bb2ea admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:50.265359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:50.266157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.266965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.267302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.268135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:50.271793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:50.326323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 916/1805] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:50.326323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75362702-2398-4768-89af-5a9bbe635707 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.343521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:50.353669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75362702-2398-4768-89af-5a9bbe635707 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:50.358566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:50.359197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.359803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.359900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.360820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:50.362719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:50.374656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:50.393550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 917/1806] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:50.393550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.402723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 890/1807] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:50 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:50.406317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.421787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18962c-e4c3-41e8-bb53-5621c3a17f74 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:50.423940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:50.562065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:50.576601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.577012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.577331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.578227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:50.585566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:50.605773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5edc5f-79be-496c-9628-cbf492161bbf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:50.617919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:50.618623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:50.618916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:50.619429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:50.751136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 918/1808] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:50 2022] POST /identity/v3/projects => generated 405 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:50.752129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.784095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:50.791667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:50.832188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 891/1809] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:50.834673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:50.854712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:50.898045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4366c5-7596-4a08-9616-ecacbe6bc9ca service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.006914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:51.016934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beff465-2b81-46e2-a512-91c74303e357 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.034016 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.034649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.034794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.035059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.035319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] PATH_INFO: `/v3/projects/95e197182adb4c9cb08c265c140e6fdc/users/7d388aeb377d40cdad0c4af57e7bd122/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.036799 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.037482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.038245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.042929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.042929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.074446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e197182adb4c9cb08c265c140e6fdc, user_id=7d388aeb377d40cdad0c4af57e7bd122, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.084268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.158668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 892/1810] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:50 2022] POST /identity/v3/users => generated 292 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:51.158668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.180836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687d228-3d58-4dfb-ada5-8cfdcf649e93 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.187344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 919/1811] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:50 2022] PUT /identity/v3/projects/95e197182adb4c9cb08c265c140e6fdc/users/7d388aeb377d40cdad0c4af57e7bd122/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:51.187344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.189087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:51.320022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.334501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.335190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.336362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.337929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.341775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.347171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6d955-11b8-4f12-a066-48356d79d9bc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.363525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.364912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.365194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.365588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.370486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:51.486478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.488337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.498281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 920/1812] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:51.511328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.532778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3b8cc-c1e3-44a6-849c-d833aa09c74b service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.541036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.542020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.542521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.543205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.544190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.547316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.572522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 921/1813] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:51.572522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.597176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e068-6425-4922-9eb8-c8a1cb72ed61 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.618959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.619949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.620805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.621302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.622827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.624612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.639539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 922/1814] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:51.646858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.661431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beeba9e-e059-4138-8afe-0184f83afba8 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.668941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.680975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.681932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.682259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.683023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.685322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.706171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 923/1815] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:51.708116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.720019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba5b08-bf0f-4b93-92a2-a9874bac86cc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.726234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.727013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.727298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.727595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] PATH_INFO: `/v3/projects/95e197182adb4c9cb08c265c140e6fdc/users/7d388aeb377d40cdad0c4af57e7bd122/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:51.729830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e197182adb4c9cb08c265c140e6fdc, user_id=7d388aeb377d40cdad0c4af57e7bd122, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:51.733147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:51.749615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 924/1816] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:51 2022] PUT /identity/v3/projects/95e197182adb4c9cb08c265c140e6fdc/users/7d388aeb377d40cdad0c4af57e7bd122/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:51.751862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.753127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 893/1817] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:51 2022] POST /identity/v3/users => generated 303 bytes in 597 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:51.754061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:51.783244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86c961-fbe6-4807-84ca-9a5ea371d0a3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.940020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bf27b-7e8e-45db-a8d2-16a4c8259bc7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:51.947203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f28f47-4e4e-4854-8931-d76a1aab7913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:51.962672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f28f47-4e4e-4854-8931-d76a1aab7913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:51.963072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f28f47-4e4e-4854-8931-d76a1aab7913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:51.963432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f28f47-4e4e-4854-8931-d76a1aab7913 admin admin] PATH_INFO: `/v3/projects/b197c587b49f40bdafd121f842b9a8a9/users/15aa20f321504b4a930ebf503147c777/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.005172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f28f47-4e4e-4854-8931-d76a1aab7913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b197c587b49f40bdafd121f842b9a8a9, user_id=15aa20f321504b4a930ebf503147c777, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:52.009633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220a987-e3a2-44c3-a287-6dd6213dd157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:52.010350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f28f47-4e4e-4854-8931-d76a1aab7913 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:52.010522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220a987-e3a2-44c3-a287-6dd6213dd157 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.010922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220a987-e3a2-44c3-a287-6dd6213dd157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.011631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220a987-e3a2-44c3-a287-6dd6213dd157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.013140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220a987-e3a2-44c3-a287-6dd6213dd157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:52.015325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220a987-e3a2-44c3-a287-6dd6213dd157 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:52.033842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 894/1818] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:52.036568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44783-cd47-42d3-8273-6dfac889ed87 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.036896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44783-cd47-42d3-8273-6dfac889ed87 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.037454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb44783-cd47-42d3-8273-6dfac889ed87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.077003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9cb44783-cd47-42d3-8273-6dfac889ed87 None None] MFA Rules not processed for user `7d388aeb377d40cdad0c4af57e7bd122`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:52.078383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9cb44783-cd47-42d3-8273-6dfac889ed87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:52.095381 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 925/1819] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:51 2022] PUT /identity/v3/projects/b197c587b49f40bdafd121f842b9a8a9/users/15aa20f321504b4a930ebf503147c777/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:52.099062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755dcb-095d-4f75-aa1a-806e4c8455e6 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.100995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755dcb-095d-4f75-aa1a-806e4c8455e6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.101126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7755dcb-095d-4f75-aa1a-806e4c8455e6 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.105021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 926/1820] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:39:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:52.111060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755dcb-095d-4f75-aa1a-806e4c8455e6 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:52.126100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:52.276597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 895/1821] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:52 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:52.278743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8c7a6-e5b1-4cde-b6d9-0a260db5c478 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.281196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8c7a6-e5b1-4cde-b6d9-0a260db5c478 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.281489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8c7a6-e5b1-4cde-b6d9-0a260db5c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.397271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7755dcb-095d-4f75-aa1a-806e4c8455e6 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:52.418653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:52.438031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:52.444256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.446026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.446423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.447814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:52.449034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a6d8c7a6-e5b1-4cde-b6d9-0a260db5c478 None None] MFA Rules not processed for user `15aa20f321504b4a930ebf503147c777`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:52.450817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:52.456901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a6d8c7a6-e5b1-4cde-b6d9-0a260db5c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:52.499761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 927/1822] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:52.505584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:52.675671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 896/1823] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:52.675671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064303c2-baf1-4fe9-a4d1-669e3250adf0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.675671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064303c2-baf1-4fe9-a4d1-669e3250adf0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.675671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064303c2-baf1-4fe9-a4d1-669e3250adf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.700139 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be27b70-bb46-4309-ac30-c9cb45016e57 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:52.717855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:52.718831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.719029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.719364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.726708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-064303c2-baf1-4fe9-a4d1-669e3250adf0 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:52.727945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-064303c2-baf1-4fe9-a4d1-669e3250adf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:52.890367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:52.894605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:52.901644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 897/1824] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:39:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:52.902618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064303c2-baf1-4fe9-a4d1-669e3250adf0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:52.923693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 928/1825] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:52 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:52.927473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:52.929376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064303c2-baf1-4fe9-a4d1-669e3250adf0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:52.939379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01acfbf-b1a7-48a4-b0a6-be839e71eb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:52.940681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01acfbf-b1a7-48a4-b0a6-be839e71eb0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.941467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01acfbf-b1a7-48a4-b0a6-be839e71eb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.942269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01acfbf-b1a7-48a4-b0a6-be839e71eb0f admin admin] PATH_INFO: `/v3/projects/338b5c03b489454e93e457184857d6a3/users/f826442d6fa34023bdcf44ce82e0242b/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.956164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb209c9e-f4b1-4660-95c5-38354bd28ea0 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:52.966995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:52.967699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:52.967979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:52.968250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:52.968921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:52.970918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:52.979993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01acfbf-b1a7-48a4-b0a6-be839e71eb0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=338b5c03b489454e93e457184857d6a3, user_id=f826442d6fa34023bdcf44ce82e0242b, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:52.985419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01acfbf-b1a7-48a4-b0a6-be839e71eb0f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:53.116213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 929/1826] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:52 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:53.117533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 898/1827] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:52 2022] PUT /identity/v3/projects/338b5c03b489454e93e457184857d6a3/users/f826442d6fa34023bdcf44ce82e0242b/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:53.128564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:53.129554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:53.130166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:53.130621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:53.184732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] MFA Rules not processed for user `f826442d6fa34023bdcf44ce82e0242b`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:53.184732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:53.259461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:53.308292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d1e68-ad4a-4e3a-b63c-b02e46e14c30 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:53.325387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:53.326067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:53.327035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:53.327035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:53.327577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:53.330360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:53.366559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 899/1828] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:53.370821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:53.370821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:53.417576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:53.437365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 930/1829] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:53 2022] POST /identity/v3/users => generated 296 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:53.439391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:53.584870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cecc91-8729-46e2-98bb-79e4233c7418 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:53.608072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:53.609628 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:53.612469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:53.618958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:53.618958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:53.637675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaae19-4c52-4c4d-88c5-0e05d32889f3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:53.658992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:53.659901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:53.661033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:53.662011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:53.683088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:53.686845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:53.712832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 900/1830] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:53.713730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:53.750827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d29d0c-ab20-4709-9abe-c26b901e5faf service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:53.771757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:53.772426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:53.772713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:53.772983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:53.773662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:53.775621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:53.825430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:53.827047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:53.852306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 931/1831] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:53 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:53.853386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:53.900637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e524-6baf-488d-ab0a-d638fcb8410d service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:53.909358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da888944-3b07-4d1f-a64b-3a273729d1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:53.909996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da888944-3b07-4d1f-a64b-3a273729d1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:53.910324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da888944-3b07-4d1f-a64b-3a273729d1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:53.910807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da888944-3b07-4d1f-a64b-3a273729d1e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:53.912323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da888944-3b07-4d1f-a64b-3a273729d1e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:53.914173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da888944-3b07-4d1f-a64b-3a273729d1e4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:53.935538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 932/1832] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:53.935698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 901/1833] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:39:53 2022] POST /identity/v3/projects => generated 403 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:53.936140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43058a-4514-4fd7-9ab1-8e4b06ec5bc7 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:53.936140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43058a-4514-4fd7-9ab1-8e4b06ec5bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:53.936140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f43058a-4514-4fd7-9ab1-8e4b06ec5bc7 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:53.937026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:53.974729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 933/1834] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:39:53 2022] GET /identity => generated 270 bytes in 41 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:39:53.976766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43058a-4514-4fd7-9ab1-8e4b06ec5bc7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:54.121917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d36ee-ce1f-4d7e-81d4-29e36e53c0c0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:54.128149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:54.128971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.130572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.130942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] PATH_INFO: `/v3/projects/13279820d63c4db98f7194c262c73e7b/users/a9b2c4ad0ecf42ba8e207684565955df/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.159350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f43058a-4514-4fd7-9ab1-8e4b06ec5bc7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:54.166350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2466d-35d9-48a2-b1d9-c3a5eda04f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:54.173428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2466d-35d9-48a2-b1d9-c3a5eda04f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.173428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2466d-35d9-48a2-b1d9-c3a5eda04f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.173428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2466d-35d9-48a2-b1d9-c3a5eda04f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.173428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2466d-35d9-48a2-b1d9-c3a5eda04f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:54.173428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2466d-35d9-48a2-b1d9-c3a5eda04f62 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:54.189938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13279820d63c4db98f7194c262c73e7b, user_id=a9b2c4ad0ecf42ba8e207684565955df, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:54.206068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:54.263992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 934/1835] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:39:53 2022] POST /identity/v3/users => generated 308 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:39:54.265006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d0ba-ec9c-4e07-ba23-959c8cb07c2c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.265287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d0ba-ec9c-4e07-ba23-959c8cb07c2c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.265561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1d0ba-ec9c-4e07-ba23-959c8cb07c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.308829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ded1d0ba-ec9c-4e07-ba23-959c8cb07c2c None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:54.310560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ded1d0ba-ec9c-4e07-ba23-959c8cb07c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:54.319891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 902/1836] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:53 2022] PUT /identity/v3/projects/13279820d63c4db98f7194c262c73e7b/users/a9b2c4ad0ecf42ba8e207684565955df/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:54.320749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:54.393578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 935/1837] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:39:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:54.394441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d0ba-ec9c-4e07-ba23-959c8cb07c2c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:54.484214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc26dda-49c9-4e88-a102-59fab3eb4768 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:54.495225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:54.495868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.496157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.496434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.497140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:54.498809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:54.505296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:54.520185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 903/1838] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:54.523987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:54.561493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d5197-636c-4007-ba4e-81af84da1d06 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:54.589374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:54.598187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.598187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.598187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] PATH_INFO: `/v3/projects/3c5bf18ba4524698ac47bf3e8a741b52/users/25a1b2181491457a82b1e3a0455cbe39/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.598741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1d0ba-ec9c-4e07-ba23-959c8cb07c2c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:54.616922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:54.622510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.622946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.623209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.623939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:54.629665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:54.643636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5bf18ba4524698ac47bf3e8a741b52, user_id=25a1b2181491457a82b1e3a0455cbe39, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:54.647745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:54.657415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 936/1839] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:39:54.663056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:54.664129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:54.686030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab1a8a-eee7-49d5-ab1a-7d4af95aeb23 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:54.698260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cd5f1-a311-4eb5-85fc-d82b384c825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:54.699025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd5f1-a311-4eb5-85fc-d82b384c825a admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.699369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd5f1-a311-4eb5-85fc-d82b384c825a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.699715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd5f1-a311-4eb5-85fc-d82b384c825a admin admin] PATH_INFO: `/v3/projects/13279820d63c4db98f7194c262c73e7b/users/a9b2c4ad0ecf42ba8e207684565955df/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.716985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cd5f1-a311-4eb5-85fc-d82b384c825a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13279820d63c4db98f7194c262c73e7b, user_id=a9b2c4ad0ecf42ba8e207684565955df, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:54.724750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cd5f1-a311-4eb5-85fc-d82b384c825a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:54.762227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 937/1840] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:54 2022] PUT /identity/v3/projects/13279820d63c4db98f7194c262c73e7b/users/a9b2c4ad0ecf42ba8e207684565955df/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:54.793223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b106-f2bf-4438-9625-884b3d96f553 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:54.795341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b106-f2bf-4438-9625-884b3d96f553 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:54.795761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b106-f2bf-4438-9625-884b3d96f553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:54.844200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 904/1841] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:54 2022] PUT /identity/v3/projects/3c5bf18ba4524698ac47bf3e8a741b52/users/25a1b2181491457a82b1e3a0455cbe39/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:54.849628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:54.927616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4128b106-f2bf-4438-9625-884b3d96f553 None None] MFA Rules not processed for user `a9b2c4ad0ecf42ba8e207684565955df`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:54.928671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4128b106-f2bf-4438-9625-884b3d96f553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:55.046828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b717254-58d4-404a-baff-ba49567e6e17 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:55.075851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:55.075851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:55.075851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:55.075851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:55.076193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:55.076193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:55.083628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 905/1842] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:39:55.084641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:55.088216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:55.135738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 938/1843] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:55.135738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b106-f2bf-4438-9625-884b3d96f553 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:55.168928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b106-f2bf-4438-9625-884b3d96f553 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:55.174929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:55.175636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:55.177026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:55.177326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] PATH_INFO: `/v3/projects/3c5bf18ba4524698ac47bf3e8a741b52/users/25a1b2181491457a82b1e3a0455cbe39/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:55.179543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5bf18ba4524698ac47bf3e8a741b52, user_id=25a1b2181491457a82b1e3a0455cbe39, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:55.189447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:55.213281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 939/1844] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:39:55 2022] PUT /identity/v3/projects/3c5bf18ba4524698ac47bf3e8a741b52/users/25a1b2181491457a82b1e3a0455cbe39/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:55.228657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:55.255459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70652ce2-94c2-436b-a773-abbfe2e37b53 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:55.329486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a159567-cb4c-41fb-ae76-0e079dc5c2d9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:55.344597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:55.350200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:55.350200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:55.350200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:55.383201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d391dad-5430-42b0-a587-479e1ed1ad2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:55.383845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d391dad-5430-42b0-a587-479e1ed1ad2a service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:55.384443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d391dad-5430-42b0-a587-479e1ed1ad2a service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:55.384766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d391dad-5430-42b0-a587-479e1ed1ad2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:55.396114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d391dad-5430-42b0-a587-479e1ed1ad2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:55.397849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d391dad-5430-42b0-a587-479e1ed1ad2a service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:55.418002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 906/1845] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:55.419801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794286c8-cbd3-4b4a-b2b4-29f10a5c30ee None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:55.420095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794286c8-cbd3-4b4a-b2b4-29f10a5c30ee None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:55.420377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794286c8-cbd3-4b4a-b2b4-29f10a5c30ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:55.462171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-794286c8-cbd3-4b4a-b2b4-29f10a5c30ee None None] MFA Rules not processed for user `25a1b2181491457a82b1e3a0455cbe39`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:55.463596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-794286c8-cbd3-4b4a-b2b4-29f10a5c30ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:55.511620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:55.513913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:55.561726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 940/1846] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:55.745014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 907/1847] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:55.799225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:55.876826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:55.887404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad71bf4-ffe5-4367-9ba9-6864864f252d service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:55.902023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:55.902890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:55.902890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:55.903252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:55.925740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:55.928072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:55.951737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 941/1848] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:56.128405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:56.193249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794286c8-cbd3-4b4a-b2b4-29f10a5c30ee None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:56.205443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794286c8-cbd3-4b4a-b2b4-29f10a5c30ee None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:56.210493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c248d99-ba0a-4825-ba68-3d1e548db048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:56.211402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248d99-ba0a-4825-ba68-3d1e548db048 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:56.216333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248d99-ba0a-4825-ba68-3d1e548db048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:56.216333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c248d99-ba0a-4825-ba68-3d1e548db048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:56.353409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c248d99-ba0a-4825-ba68-3d1e548db048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:56.355028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c248d99-ba0a-4825-ba68-3d1e548db048 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:56.369368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 908/1849] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:56.867932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:57.008580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e365-d03e-4e0b-9620-1c465e64eb0a service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:57.019357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:57.019357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:57.019357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:57.019357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] PATH_INFO: `/v3/users/f826442d6fa34023bdcf44ce82e0242b` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:57.021636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f826442d6fa34023bdcf44ce82e0242b)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:57.021636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:57.299132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f826442d6fa34023bdcf44ce82e0242b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:57.343291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:57.359148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f826442d6fa34023bdcf44ce82e0242b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:57.447777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f826442d6fa34023bdcf44ce82e0242b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:57.448535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f826442d6fa34023bdcf44ce82e0242b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:57.449170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f826442d6fa34023bdcf44ce82e0242b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:57.475957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 942/1850] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:56 2022] DELETE /identity/v3/users/f826442d6fa34023bdcf44ce82e0242b => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:57.482949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:57.483519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:57.483519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:57.514854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:57.516761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:57.641118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 909/1851] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:57.659494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:57.661517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:57.730801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:57.732619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:57.757906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b4fca-fa84-4940-b4cb-827a34654589 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:57.768994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:57.770665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:57.771017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:57.771642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] PATH_INFO: `/v3/users/15aa20f321504b4a930ebf503147c777` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:57.775634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=15aa20f321504b4a930ebf503147c777)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:57.779454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:57.834836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6a997-5359-4c92-8568-51fd762a1d1a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:57.839996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabb6c4-c287-4d74-80fd-63d0cf6e86dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:57.840724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabb6c4-c287-4d74-80fd-63d0cf6e86dd service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:57.841007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabb6c4-c287-4d74-80fd-63d0cf6e86dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:57.841328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabb6c4-c287-4d74-80fd-63d0cf6e86dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:57.855321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabb6c4-c287-4d74-80fd-63d0cf6e86dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:57.857691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabb6c4-c287-4d74-80fd-63d0cf6e86dd service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:57.872156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 943/1852] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:58.111777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15aa20f321504b4a930ebf503147c777'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:58.137430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15aa20f321504b4a930ebf503147c777'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:58.223083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15aa20f321504b4a930ebf503147c777'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:58.233505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15aa20f321504b4a930ebf503147c777'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:58.243992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15aa20f321504b4a930ebf503147c777'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:58.263848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 910/1853] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:57 2022] DELETE /identity/v3/users/15aa20f321504b4a930ebf503147c777 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:39:58.270997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:58.270997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:58.271345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:58.337015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:39:58.338587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:58.403130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:58.460710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 944/1854] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:39:58.488739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:58.490432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:58.613702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41de7b-3f50-4f59-b03c-c6927382df7b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:58.634245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:58.635053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:58.636575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:58.636920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:58.649681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:58.655097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:58.671546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:58.672983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:58.694292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 911/1855] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:39:58.700678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cab1a-4892-422e-81eb-ac74b6db86b7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:58.727250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:58.728387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:58.728825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:58.730389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] PATH_INFO: `/v3/projects/338b5c03b489454e93e457184857d6a3` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:58.734475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] RBAC: Authorizing `identity:delete_project(project_id=338b5c03b489454e93e457184857d6a3)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:58.744792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:58.965351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338b5c03b489454e93e457184857d6a3', 'request_id': 'req-0b708bab-6a55-4f00-a087-95f6c382197f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:58.975245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338b5c03b489454e93e457184857d6a3', 'request_id': 'req-0b708bab-6a55-4f00-a087-95f6c382197f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.045559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338b5c03b489454e93e457184857d6a3', 'request_id': 'req-0b708bab-6a55-4f00-a087-95f6c382197f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.067455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] The token cache is being invalidate because project 338b5c03b489454e93e457184857d6a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:39:59.067931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.068727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.073324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 945/1856] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:58 2022] DELETE /identity/v3/projects/338b5c03b489454e93e457184857d6a3 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:39:59.075462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:59.093513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f91194-60d6-4bfc-9ee1-d89f0137a9e6 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:59.188507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:59.189327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:59.189689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:59.190064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] PATH_INFO: `/v3/users/25a1b2181491457a82b1e3a0455cbe39` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:59.192137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=25a1b2181491457a82b1e3a0455cbe39)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:59.194783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:59.205236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:39:59.401626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25a1b2181491457a82b1e3a0455cbe39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.419268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25a1b2181491457a82b1e3a0455cbe39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.517739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25a1b2181491457a82b1e3a0455cbe39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.527274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:59.530620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:59.547585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25a1b2181491457a82b1e3a0455cbe39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.559712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25a1b2181491457a82b1e3a0455cbe39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.571506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b708bab-6a55-4f00-a087-95f6c382197f admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:59.577424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 912/1857] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:59 2022] DELETE /identity/v3/users/25a1b2181491457a82b1e3a0455cbe39 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:39:59.721752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:59.723793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:59.723793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:59.723793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] PATH_INFO: `/v3/projects/b197c587b49f40bdafd121f842b9a8a9` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:59.732166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b197c587b49f40bdafd121f842b9a8a9)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:59.732166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:59.766816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:39:59.768273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:39:59.790791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eae420-9c9b-42cf-9acf-a371b6bb20ed admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:39:59.803267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:39:59.806674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:39:59.806674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:39:59.806674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] PATH_INFO: `/v3/users/8cc50f40c890475f9b3ba03c51b74a72` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:39:59.806674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc50f40c890475f9b3ba03c51b74a72)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:39:59.808843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:39:59.876430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b197c587b49f40bdafd121f842b9a8a9', 'request_id': 'req-1db7d972-8ccc-4b0d-816d-8db9a9799635'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:39:59.891994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b197c587b49f40bdafd121f842b9a8a9', 'request_id': 'req-1db7d972-8ccc-4b0d-816d-8db9a9799635'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.015691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b197c587b49f40bdafd121f842b9a8a9', 'request_id': 'req-1db7d972-8ccc-4b0d-816d-8db9a9799635'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.023747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] The token cache is being invalidate because project b197c587b49f40bdafd121f842b9a8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:00.024038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.024883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.026665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 946/1858] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:39:59 2022] DELETE /identity/v3/projects/b197c587b49f40bdafd121f842b9a8a9 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:00.030847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:00.032189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:00.099008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc50f40c890475f9b3ba03c51b74a72'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.127203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d972-8ccc-4b0d-816d-8db9a9799635 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:00.133810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc50f40c890475f9b3ba03c51b74a72'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.135045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:00.135717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.136881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.138243 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] PATH_INFO: `/v3/users/a9b2c4ad0ecf42ba8e207684565955df` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.141543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9b2c4ad0ecf42ba8e207684565955df)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:00.144091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:00.204456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc50f40c890475f9b3ba03c51b74a72'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.213573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc50f40c890475f9b3ba03c51b74a72'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.214476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc50f40c890475f9b3ba03c51b74a72'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.220513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 913/1859] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:39:59 2022] DELETE /identity/v3/users/8cc50f40c890475f9b3ba03c51b74a72 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:00.221317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:00.222725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:00.292633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8bbdc-0ee8-4379-8399-5ba1f8d46cb8 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:00.297980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:00.298665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.298943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.299214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] PATH_INFO: `/v3/projects/3c5bf18ba4524698ac47bf3e8a741b52` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.300392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5bf18ba4524698ac47bf3e8a741b52)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:00.303039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:00.350582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9b2c4ad0ecf42ba8e207684565955df'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.362238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9b2c4ad0ecf42ba8e207684565955df'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.433080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5bf18ba4524698ac47bf3e8a741b52', 'request_id': 'req-f1946874-8e47-4565-ba7b-4ba443b6bd0d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9b2c4ad0ecf42ba8e207684565955df'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9b2c4ad0ecf42ba8e207684565955df'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.439177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2b5cf787-2111-4933-9f72-fa8971c975e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9b2c4ad0ecf42ba8e207684565955df'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.442688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5bf18ba4524698ac47bf3e8a741b52', 'request_id': 'req-f1946874-8e47-4565-ba7b-4ba443b6bd0d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.452640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 947/1860] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:00 2022] DELETE /identity/v3/users/a9b2c4ad0ecf42ba8e207684565955df => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:00.453263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5bf18ba4524698ac47bf3e8a741b52', 'request_id': 'req-f1946874-8e47-4565-ba7b-4ba443b6bd0d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.454136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.454490 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.454760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.484625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:00.485835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:00.512477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:00.531712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] The token cache is being invalidate because project 3c5bf18ba4524698ac47bf3e8a741b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:00.537131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.537131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1946874-8e47-4565-ba7b-4ba443b6bd0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:00.538904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 914/1861] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:00 2022] DELETE /identity/v3/projects/3c5bf18ba4524698ac47bf3e8a741b52 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:00.540616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54060869-7919-4685-9cb9-6dca466ac3b6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.541058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54060869-7919-4685-9cb9-6dca466ac3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.541972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54060869-7919-4685-9cb9-6dca466ac3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.576072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 948/1862] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:00.586724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:00.587692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-54060869-7919-4685-9cb9-6dca466ac3b6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:00.588678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:00.589316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54060869-7919-4685-9cb9-6dca466ac3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:00.601827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 915/1863] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:00.617301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf323c-018d-4fda-818c-d4dcfbae02e1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.618993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf323c-018d-4fda-818c-d4dcfbae02e1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.618993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf323c-018d-4fda-818c-d4dcfbae02e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.641084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c6cf323c-018d-4fda-818c-d4dcfbae02e1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:00.642350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c6cf323c-018d-4fda-818c-d4dcfbae02e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:00.654121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 916/1864] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:00.658040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf323c-018d-4fda-818c-d4dcfbae02e1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:00.674316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf323c-018d-4fda-818c-d4dcfbae02e1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:00.680084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:00.680704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.680979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.681304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.682004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:00.682355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:00.683940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:00.685339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330f6ec-bd22-42e0-8607-63c641ecb7cf None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:00.690378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:00.691002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.691587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.692010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.697526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 917/1865] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:40:00.702367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:00.707615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:00.709225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:00.719620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27fc8f-b326-4f11-9b96-be01f484f04a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:00.721469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 949/1866] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:00.726971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:00.727604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.727883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.728156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.729133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:00.731894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:00.891531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 918/1867] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:00 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:00.892867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:00.903216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8771a4-a2fd-457c-b9ef-7183cabca80c service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:00.903447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:00.962519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:00.963149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:00.963434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:00.963709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:00.964402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:00.966516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.041236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.043909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.066308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 950/1868] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:00 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:01.068047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b670d-d59c-4fae-9ce8-cac6ec7df6d2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.071109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.073291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.073949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.074301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.074579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] PATH_INFO: `/v3/projects/13279820d63c4db98f7194c262c73e7b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.075743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] RBAC: Authorizing `identity:delete_project(project_id=13279820d63c4db98f7194c262c73e7b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.080899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.087983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57da29c-4a4b-423d-bc12-2d81bc2d2ddf admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.109838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.111126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.112830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.113883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.115261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.118877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.131955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 951/1869] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:01.135851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.155645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908cbcc-a279-4160-951f-a9dbbbfde462 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.163269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.163888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.164168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.164440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.165157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.166860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.184226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 952/1870] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:01.187277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.197631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70c753-f2fa-4fe3-aa24-e31d45238a93 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.202844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.203792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.204155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.204594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] PATH_INFO: `/v3/projects/0811135a85e340a99da66a7242f889ab/users/26460f0ed2554c9f958bb2c49a526e2a/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.228367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13279820d63c4db98f7194c262c73e7b', 'request_id': 'req-7e8d96d4-3694-49a4-a529-9144a59b874d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.230251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0811135a85e340a99da66a7242f889ab, user_id=26460f0ed2554c9f958bb2c49a526e2a, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.235817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13279820d63c4db98f7194c262c73e7b', 'request_id': 'req-7e8d96d4-3694-49a4-a529-9144a59b874d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.239021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.247718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13279820d63c4db98f7194c262c73e7b', 'request_id': 'req-7e8d96d4-3694-49a4-a529-9144a59b874d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.355323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 953/1871] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:01 2022] PUT /identity/v3/projects/0811135a85e340a99da66a7242f889ab/users/26460f0ed2554c9f958bb2c49a526e2a/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:01.355521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] The token cache is being invalidate because project 13279820d63c4db98f7194c262c73e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:01.355521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.358307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e8d96d4-3694-49a4-a529-9144a59b874d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.361804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 919/1872] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:01 2022] DELETE /identity/v3/projects/13279820d63c4db98f7194c262c73e7b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:01.364894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.367185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4eecd-5103-40ee-8c1f-c30d04802c82 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.367580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4eecd-5103-40ee-8c1f-c30d04802c82 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.367962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4eecd-5103-40ee-8c1f-c30d04802c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.372041 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.394211 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-95c4eecd-5103-40ee-8c1f-c30d04802c82 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:01.396285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-95c4eecd-5103-40ee-8c1f-c30d04802c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.484472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45790fd-49f2-4ea9-9bd4-588f793c5e85 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.490231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.491313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.491922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.492408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] PATH_INFO: `/v3/projects/398e5916da014b0683dd7ba1664d49a0` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.493906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=398e5916da014b0683dd7ba1664d49a0)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.496143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.496256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 920/1873] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:01.497438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1429e-c963-40bc-92bf-6f42295e5bb3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.497719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1429e-c963-40bc-92bf-6f42295e5bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.497988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1429e-c963-40bc-92bf-6f42295e5bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.529913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d2a1429e-c963-40bc-92bf-6f42295e5bb3 None None] MFA Rules not processed for user `26460f0ed2554c9f958bb2c49a526e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:01.531997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d2a1429e-c963-40bc-92bf-6f42295e5bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.616494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 921/1874] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:01.617652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.617941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.618241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.639272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:01.641447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '398e5916da014b0683dd7ba1664d49a0', 'request_id': 'req-26dfa101-ce41-4fda-8c03-21f18e13dba3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.642175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.650777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '398e5916da014b0683dd7ba1664d49a0', 'request_id': 'req-26dfa101-ce41-4fda-8c03-21f18e13dba3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.707561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 922/1875] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:01.708596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.711150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.723250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325c7f-9d2a-4a40-9148-7e1262c9437b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.728684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.729301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.729577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.729844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.730552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.732542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.736232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '398e5916da014b0683dd7ba1664d49a0', 'request_id': 'req-26dfa101-ce41-4fda-8c03-21f18e13dba3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.748445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] The token cache is being invalidate because project 398e5916da014b0683dd7ba1664d49a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:01.748738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.749408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:01.750994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 954/1876] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:01 2022] DELETE /identity/v3/projects/398e5916da014b0683dd7ba1664d49a0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:01.754726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.756079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.770826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa101-ce41-4fda-8c03-21f18e13dba3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.776887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.777411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.777683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.777954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.778726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:01.779030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.780694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.795565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 955/1877] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:01.796485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.801602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:01.816869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2a13-fbe3-4ed9-81c6-a248c0fc69e4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.821586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.822286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.822572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.822847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] PATH_INFO: `/v3/users/8f5f98eb8fd142e1b278ef50dd0dec8e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.824029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5f98eb8fd142e1b278ef50dd0dec8e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.826453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:01.888605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 923/1878] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:01 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:01.888605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:01.986047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389e47c-b5e3-44ea-82de-723f1cacbf6f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:01.986047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:01.986047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:01.986047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:01.986379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:01.986379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:01.987571 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.040125 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:02.078720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 924/1879] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:01 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:02.079838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.083161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:02.106279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5f98eb8fd142e1b278ef50dd0dec8e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:02.119705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5f98eb8fd142e1b278ef50dd0dec8e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:02.154519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac797026-66f3-4df1-aac5-27a6bf10d963 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.159272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.159925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.160207 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.160491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.161145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.163119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.216804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5f98eb8fd142e1b278ef50dd0dec8e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:02.217422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5f98eb8fd142e1b278ef50dd0dec8e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:02.218034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5f98eb8fd142e1b278ef50dd0dec8e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:02.230726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 956/1880] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:01 2022] DELETE /identity/v3/users/8f5f98eb8fd142e1b278ef50dd0dec8e => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:02.231690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.233181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:02.253568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 925/1881] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:02 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:02.256894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.260180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:02.278244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3a4ff-e16a-4342-9bdb-d6c47e7cb1fe admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.304403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fb0a4-69fa-4935-8934-c58d0b7067cc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.310645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.310999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.311278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.311671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.312569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.314634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.315402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:02.353476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.354106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.354443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.354833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.355465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.357056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.369506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 926/1882] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:02.373437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.384940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d747a-c713-4176-883f-6e06566cc2d5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.395731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.395731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.395731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.395731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] PATH_INFO: `/v3/projects/787f048f62ef4604913e0406b7db1de8/users/945e9111444a413cb26812291e59e075/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.406776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787f048f62ef4604913e0406b7db1de8, user_id=945e9111444a413cb26812291e59e075, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.409467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.432256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 957/1883] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:02 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:02.433331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.443738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0e7f0-59c3-46ae-930a-9bb82297cc4e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.448490 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.449172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.449462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.449858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.450626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.452206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.462936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 958/1884] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:02.466764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.476570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8632e-c8cd-4c58-8981-0ad1b754b4b7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.481100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.481692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.481978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.482281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.487548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.487548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.499165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 959/1885] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:02.500064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.511279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1c43-73e4-457f-a7e6-4fb35dc10dc2 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.516142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130bc62-fdcb-4d03-8be6-8d5c7bf203de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.516866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130bc62-fdcb-4d03-8be6-8d5c7bf203de admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.517217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130bc62-fdcb-4d03-8be6-8d5c7bf203de admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.517558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130bc62-fdcb-4d03-8be6-8d5c7bf203de admin admin] PATH_INFO: `/v3/projects/66825c1377de4136bda787d5d4e78c23/users/153baa095b9a40028f21daa9d8d916b8/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.534726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130bc62-fdcb-4d03-8be6-8d5c7bf203de admin admin] RBAC: Authorizing `identity:create_grant(project_id=66825c1377de4136bda787d5d4e78c23, user_id=153baa095b9a40028f21daa9d8d916b8, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.538291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130bc62-fdcb-4d03-8be6-8d5c7bf203de admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.552923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 927/1886] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:02 2022] PUT /identity/v3/projects/787f048f62ef4604913e0406b7db1de8/users/945e9111444a413cb26812291e59e075/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:02.556586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.635456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452c698-c66b-4a49-a295-e9f5ac107761 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.640961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 960/1887] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:02 2022] PUT /identity/v3/projects/66825c1377de4136bda787d5d4e78c23/users/153baa095b9a40028f21daa9d8d916b8/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:02.652701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c8951-b22e-4785-bdba-a44c4735ab9d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.653495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c8951-b22e-4785-bdba-a44c4735ab9d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.653495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c8951-b22e-4785-bdba-a44c4735ab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.663665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:02.692199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-452c8951-b22e-4785-bdba-a44c4735ab9d None None] MFA Rules not processed for user `153baa095b9a40028f21daa9d8d916b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:02.693417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-452c8951-b22e-4785-bdba-a44c4735ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:02.702567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.703313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.703598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.703872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.704587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.706300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.717125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 928/1888] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:02.721309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.731085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc9eab-4a03-4269-93bf-89701041b2c5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.735689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.736337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.736620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.736898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] PATH_INFO: `/v3/projects/787f048f62ef4604913e0406b7db1de8/users/945e9111444a413cb26812291e59e075/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.738891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787f048f62ef4604913e0406b7db1de8, user_id=945e9111444a413cb26812291e59e075, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.741591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.755868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 929/1889] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:02 2022] PUT /identity/v3/projects/787f048f62ef4604913e0406b7db1de8/users/945e9111444a413cb26812291e59e075/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:02.756720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.758345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:02.759933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 961/1890] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:02.772515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.773261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.773261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.803247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] MFA Rules not processed for user `945e9111444a413cb26812291e59e075`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:02.804523 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:02.832567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82c77-94b8-4364-b3be-09a54a51f455 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.837552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.838255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.838533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.838805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] PATH_INFO: `/v3/projects/ca52792fdd7148589ae321c1309430f7` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.839976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca52792fdd7148589ae321c1309430f7)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:02.841861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:02.874522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 962/1891] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:02.880456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:02.881699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:02.890260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:02.939723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ecc80-4992-42dd-b9a5-34a64bab32bc None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:02.943981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:02.944598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:02.944875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:02.945145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:02.977545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca52792fdd7148589ae321c1309430f7', 'request_id': 'req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:02.983485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca52792fdd7148589ae321c1309430f7', 'request_id': 'req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:02.990431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca52792fdd7148589ae321c1309430f7', 'request_id': 'req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.010272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.011884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.022213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 963/1892] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:03.023209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.024531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.042360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e64f8-5397-48ad-9f3b-5eb59e4f3216 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.079621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] The token cache is being invalidate because project ca52792fdd7148589ae321c1309430f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:03.080033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.080773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6bbf95d-2c52-4e66-8dbc-e138cbe90321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.083189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 930/1893] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:02 2022] DELETE /identity/v3/projects/ca52792fdd7148589ae321c1309430f7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:03.087900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cfe35-140a-431d-bfb7-745417f33613 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.088644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cfe35-140a-431d-bfb7-745417f33613 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.089004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cfe35-140a-431d-bfb7-745417f33613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.101393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.102413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.102859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.103205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.105286 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.115809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-660cfe35-140a-431d-bfb7-745417f33613 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:03.116787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-660cfe35-140a-431d-bfb7-745417f33613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.184955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.186677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.188938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 931/1894] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:03.197819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7050a82-9607-409f-b108-667db4398631 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.199064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7050a82-9607-409f-b108-667db4398631 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.199350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7050a82-9607-409f-b108-667db4398631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.207118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 964/1895] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:03.221957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e7050a82-9607-409f-b108-667db4398631 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:03.223431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e7050a82-9607-409f-b108-667db4398631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.231828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.236281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 932/1896] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:03.236389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.239391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7050a82-9607-409f-b108-667db4398631 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.251603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c1381-7670-4884-8b93-3059d768445c service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.251750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7050a82-9607-409f-b108-667db4398631 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.257156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.257306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.257842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.257979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.258123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.258283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.258427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.258558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] PATH_INFO: `/v3/users/26460f0ed2554c9f958bb2c49a526e2a` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.265736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:03.265736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.266116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26460f0ed2554c9f958bb2c49a526e2a)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.266229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.266343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.285358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 933/1897] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:40:03.286512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.297365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7e892-440b-4bd0-9694-235e9ffb64e2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.303658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.304311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.304593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.304869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.305613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.307544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.393677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 934/1898] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:03 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:03.395770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.461253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8e6e3-9882-4884-9375-2928fc6626af admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.467060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.467687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.467969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.468238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.468903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.470947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.475788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26460f0ed2554c9f958bb2c49a526e2a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.490591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26460f0ed2554c9f958bb2c49a526e2a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.554261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 935/1899] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:03 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:03.557815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.568165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bc097-df3e-471c-9983-a36e967b61a2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.573159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.573880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.574282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.574643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.575416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.577077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.588601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 936/1900] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:03.592474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.602055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a90686-217f-4dd2-9abb-ad5b805bff7d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.604591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26460f0ed2554c9f958bb2c49a526e2a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.605935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26460f0ed2554c9f958bb2c49a526e2a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.606986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-552a3064-6809-4056-9117-1b86d769b939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26460f0ed2554c9f958bb2c49a526e2a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:03.607777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.608396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.608747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.608953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.609645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.611283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.623113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 937/1901] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:03.630404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 965/1902] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:03 2022] DELETE /identity/v3/users/26460f0ed2554c9f958bb2c49a526e2a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:03.631679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.633509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.634157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.634473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.634768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.647228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:03.648113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7422b-d383-455f-9e12-0cceaae69177 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.657930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:03.659204 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.709013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd9da4-3977-4def-b3fe-b13e9888a606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.709701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9da4-3977-4def-b3fe-b13e9888a606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.709986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9da4-3977-4def-b3fe-b13e9888a606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.710308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9da4-3977-4def-b3fe-b13e9888a606 admin admin] PATH_INFO: `/v3/projects/2e106d38d8b84299808e60a1deb59480/users/fe245f6195114b8ab1ec98cd683876f2/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.723554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 966/1903] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:03.728090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd9da4-3977-4def-b3fe-b13e9888a606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e106d38d8b84299808e60a1deb59480, user_id=fe245f6195114b8ab1ec98cd683876f2, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.733091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd9da4-3977-4def-b3fe-b13e9888a606 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.740823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.742576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.795451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 938/1904] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:03 2022] PUT /identity/v3/projects/2e106d38d8b84299808e60a1deb59480/users/fe245f6195114b8ab1ec98cd683876f2/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:03.800310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:03.806497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27138e33-7a00-4e5b-b639-6d1fe320d29d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.806704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aef6a1-0971-465a-a555-e6a1c9b33c0a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.807637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aef6a1-0971-465a-a555-e6a1c9b33c0a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.807997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aef6a1-0971-465a-a555-e6a1c9b33c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.837920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-79aef6a1-0971-465a-a555-e6a1c9b33c0a None None] MFA Rules not processed for user `fe245f6195114b8ab1ec98cd683876f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:03.839372 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-79aef6a1-0971-465a-a555-e6a1c9b33c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:03.894346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.895159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.895453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.895736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.909836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 939/1905] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:03.915653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aef6a1-0971-465a-a555-e6a1c9b33c0a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:03.961005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:03.962868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:03.973303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 967/1906] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:03.991934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aef6a1-0971-465a-a555-e6a1c9b33c0a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:03.996407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:03.997031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:03.997323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:03.997594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:03.998320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.000194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:04.159570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 940/1907] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:03 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:04.161236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.171218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70742fb1-dfec-486e-8b79-d1ce44e7034a service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:04.232756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:04.232756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.232756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.232756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.233485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.233485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:04.341991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 968/1908] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:04 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:04.344291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.361690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1a97d-1abb-429a-8649-9ecd05ccf9ec admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:04.367443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:04.368070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.368353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.368736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.369681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.371120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:04.384682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 941/1909] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:04.386270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.397707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c6c11-2505-4211-89d1-60c0cd611049 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:04.403942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:04.404847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.405318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.405812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.406990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.409037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:04.434162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 969/1910] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:04.438347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.448991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97397-efcc-4771-9b98-dc5c559f0793 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:04.453591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f5c40-116d-4844-bb2d-7b3033ff14d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:04.454293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5c40-116d-4844-bb2d-7b3033ff14d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.460285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5c40-116d-4844-bb2d-7b3033ff14d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.460285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5c40-116d-4844-bb2d-7b3033ff14d9 admin admin] PATH_INFO: `/v3/projects/185895c0893c4617a435e843e971582f/users/8902e533927b4b598cb5a717d5156173/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.470390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f5c40-116d-4844-bb2d-7b3033ff14d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=185895c0893c4617a435e843e971582f, user_id=8902e533927b4b598cb5a717d5156173, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.473632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f5c40-116d-4844-bb2d-7b3033ff14d9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:04.572508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 942/1911] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:04 2022] PUT /identity/v3/projects/185895c0893c4617a435e843e971582f/users/8902e533927b4b598cb5a717d5156173/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:04.578012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.580885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:04.591947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc57de-0d35-49c6-ad94-121108c1864e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.592224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc57de-0d35-49c6-ad94-121108c1864e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.592537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc57de-0d35-49c6-ad94-121108c1864e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.608880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dfdd2-5b8a-4003-afbb-ae285d978e40 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:04.626168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b5dc57de-0d35-49c6-ad94-121108c1864e None None] MFA Rules not processed for user `8902e533927b4b598cb5a717d5156173`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:04.627470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b5dc57de-0d35-49c6-ad94-121108c1864e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:04.694377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:04.694377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.694377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.694377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] PATH_INFO: `/v3/projects/0811135a85e340a99da66a7242f889ab` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.694722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0811135a85e340a99da66a7242f889ab)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.694722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:04.729773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 943/1912] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:04.730795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc57de-0d35-49c6-ad94-121108c1864e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.746726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc57de-0d35-49c6-ad94-121108c1864e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:04.751615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:04.752276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.752633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.753051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.754329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.756245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:04.839949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0811135a85e340a99da66a7242f889ab', 'request_id': 'req-480c9719-3127-46c9-9563-790b8fdfcc8d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:04.845826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:04.853686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0811135a85e340a99da66a7242f889ab', 'request_id': 'req-480c9719-3127-46c9-9563-790b8fdfcc8d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:04.858072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 944/1913] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:04 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:04.859381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.914137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0811135a85e340a99da66a7242f889ab', 'request_id': 'req-480c9719-3127-46c9-9563-790b8fdfcc8d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:04.935052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] The token cache is being invalidate because project 0811135a85e340a99da66a7242f889ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:04.944187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:04.944187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:04.944512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae382d6-006f-4542-b300-100fffd4429c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:04.947622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 970/1914] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:04 2022] DELETE /identity/v3/projects/0811135a85e340a99da66a7242f889ab => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:04.948486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:04.953301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:04.954299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:04.955274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:04.955725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:04.956212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:04.958034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:04.960445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:05.045908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c9719-3127-46c9-9563-790b8fdfcc8d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:05.053086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:05.053086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.053086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.053349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] PATH_INFO: `/v3/users/945e9111444a413cb26812291e59e075` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.054872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=945e9111444a413cb26812291e59e075)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:05.057534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:05.130618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 945/1915] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:04 2022] POST /identity/v3/users => generated 296 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:05.134306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:05.136282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:05.149370 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9082c42-cfbc-4550-b2fa-e2eafab40e3b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:05.154952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:05.155706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.159305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.159305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.159305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:05.159305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:05.169732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 946/1916] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:05.173774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:05.181930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:05.185610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caa296-62c1-4d5d-ad7b-037bce3034bd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:05.190323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:05.190982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.191288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.191743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] PATH_INFO: `/v3/projects/aac629981b764f90884a041d600d2405/users/f8ee3f8d323a4d9db6e19e6cd8ab2760/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.207340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac629981b764f90884a041d600d2405, user_id=f8ee3f8d323a4d9db6e19e6cd8ab2760, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:05.212349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:05.273004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945e9111444a413cb26812291e59e075'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:05.289532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945e9111444a413cb26812291e59e075'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:05.327522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 947/1917] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:05 2022] PUT /identity/v3/projects/aac629981b764f90884a041d600d2405/users/f8ee3f8d323a4d9db6e19e6cd8ab2760/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:05.327522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:05.395027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cce20-617d-4340-96c6-99ec517735b2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:05.399802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:05.400466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.400743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.401036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.401726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:05.403351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:05.405107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945e9111444a413cb26812291e59e075'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:05.405742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945e9111444a413cb26812291e59e075'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:05.406362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a880b516-d315-4ad9-ac59-1db8c454c345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945e9111444a413cb26812291e59e075'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:05.417525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 948/1918] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:05.418758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:05.419895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 971/1919] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:04 2022] DELETE /identity/v3/users/945e9111444a413cb26812291e59e075 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:05.420345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:05.435149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebbc69-b19d-4fac-824f-2058138ee898 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:05.534702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:05.535428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.535785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.535992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] PATH_INFO: `/v3/projects/aac629981b764f90884a041d600d2405/users/f8ee3f8d323a4d9db6e19e6cd8ab2760/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.538290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac629981b764f90884a041d600d2405, user_id=f8ee3f8d323a4d9db6e19e6cd8ab2760, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:05.541960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:05.555051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 949/1920] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:05 2022] PUT /identity/v3/projects/aac629981b764f90884a041d600d2405/users/f8ee3f8d323a4d9db6e19e6cd8ab2760/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:05.566066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.567021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.567354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.598269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] MFA Rules not processed for user `f8ee3f8d323a4d9db6e19e6cd8ab2760`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:05.599483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:05.687164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 972/1921] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:05.700269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:05.707806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:05.771383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03254c82-5fc8-4e55-953a-56844ed30b61 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:05.775307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:05.779916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.779916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.779916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.788494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:05.789866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:05.800262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 950/1922] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:05.839807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:05.924281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:05.927017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:05.931375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:05.933757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:05.950038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c27c6-a6e5-4d6c-95df-4cb17939c04a service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:05.956040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3eda51-8806-4130-ab4b-9ce58740bada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:05.957809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eda51-8806-4130-ab4b-9ce58740bada service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:05.958410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eda51-8806-4130-ab4b-9ce58740bada service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:05.958650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eda51-8806-4130-ab4b-9ce58740bada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:05.966906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fc3eda51-8806-4130-ab4b-9ce58740bada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:06.011477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:06.057065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a1fda-926f-488e-aaa2-b5b2ff9c0ee6 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:06.063086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:06.073479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.074489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.075018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] PATH_INFO: `/v3/projects/787f048f62ef4604913e0406b7db1de8` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.076570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787f048f62ef4604913e0406b7db1de8)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.086241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.126632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3eda51-8806-4130-ab4b-9ce58740bada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.128498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3eda51-8806-4130-ab4b-9ce58740bada service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.141202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 951/1923] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:06.199982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787f048f62ef4604913e0406b7db1de8', 'request_id': 'req-cb7b2d41-c6b0-4e96-b242-870b7e912335'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:06.207928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787f048f62ef4604913e0406b7db1de8', 'request_id': 'req-cb7b2d41-c6b0-4e96-b242-870b7e912335'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:06.258026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787f048f62ef4604913e0406b7db1de8', 'request_id': 'req-cb7b2d41-c6b0-4e96-b242-870b7e912335'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:06.276882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] The token cache is being invalidate because project 787f048f62ef4604913e0406b7db1de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:06.277481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:06.277870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cb7b2d41-c6b0-4e96-b242-870b7e912335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:06.280360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 973/1924] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:05 2022] DELETE /identity/v3/projects/787f048f62ef4604913e0406b7db1de8 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:06.286022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e901a-be3c-45b2-a92e-d60491604388 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.297051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e901a-be3c-45b2-a92e-d60491604388 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.297461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e901a-be3c-45b2-a92e-d60491604388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.318810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ed0e901a-be3c-45b2-a92e-d60491604388 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:06.320030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ed0e901a-be3c-45b2-a92e-d60491604388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:06.385645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 952/1925] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:06.393100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208c56f-0e1e-4c92-a9be-1f1bbb002fc0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.393385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208c56f-0e1e-4c92-a9be-1f1bbb002fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.393663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208c56f-0e1e-4c92-a9be-1f1bbb002fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.418294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b208c56f-0e1e-4c92-a9be-1f1bbb002fc0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:06.418294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b208c56f-0e1e-4c92-a9be-1f1bbb002fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:06.424335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 974/1926] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:06.428926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e901a-be3c-45b2-a92e-d60491604388 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:06.438682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e901a-be3c-45b2-a92e-d60491604388 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:06.444653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:06.445276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.445558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.446958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.446958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:06.447207 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.448613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.462587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 953/1927] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:06.467092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208c56f-0e1e-4c92-a9be-1f1bbb002fc0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:06.478634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208c56f-0e1e-4c92-a9be-1f1bbb002fc0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:06.487182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:06.492335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.492621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.493017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.494493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.498642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.582042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 975/1928] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:06 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:06.586682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:06.596485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b50be-80bf-4b05-b6c1-55d1c6db699f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:06.652646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:06.653267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.653578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.653850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.654547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.656484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.778617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 954/1929] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:06 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:06.783972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:06.795440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c00b-f8ab-4167-9205-7aa352aeffe1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:06.800546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:06.801220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.801746 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.802024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.802770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.804646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.815983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 976/1930] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:06.820352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:06.831115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857bd81-803d-4fa0-9fb6-1a457bb328ee admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:06.837045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8631125-de13-404e-a725-60dc69fd96a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:06.837774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8631125-de13-404e-a725-60dc69fd96a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.838060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8631125-de13-404e-a725-60dc69fd96a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.838357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8631125-de13-404e-a725-60dc69fd96a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.839058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8631125-de13-404e-a725-60dc69fd96a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.840980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8631125-de13-404e-a725-60dc69fd96a5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.856345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 955/1931] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:06.861313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:06.879849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67bf39-2402-4dd1-bfc7-6b8a9f1b29aa admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:06.885155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:06.885925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.886454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.886793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] PATH_INFO: `/v3/projects/1963465f3fe8443a90998e8a762e4be0/users/79f168e4d95149519e1408612dd40e1f/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:06.902592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1963465f3fe8443a90998e8a762e4be0, user_id=79f168e4d95149519e1408612dd40e1f, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:06.905307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:06.987087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 977/1932] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:06 2022] PUT /identity/v3/projects/1963465f3fe8443a90998e8a762e4be0/users/79f168e4d95149519e1408612dd40e1f/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:06.998537 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42846e4b-bddc-4ec4-a4ee-d3ddf912956d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:06.998836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42846e4b-bddc-4ec4-a4ee-d3ddf912956d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:06.999108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42846e4b-bddc-4ec4-a4ee-d3ddf912956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.029423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-42846e4b-bddc-4ec4-a4ee-d3ddf912956d None None] MFA Rules not processed for user `79f168e4d95149519e1408612dd40e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:07.030693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-42846e4b-bddc-4ec4-a4ee-d3ddf912956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:07.097989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 956/1933] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:07.102568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:07.179770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006efae-75f5-43e5-892b-386cce6156ba admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:07.183724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:07.189582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:07.192053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.192053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.192939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.193226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:07.197298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:07.332759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 978/1934] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:07 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:07.334843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42846e4b-bddc-4ec4-a4ee-d3ddf912956d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:07.357425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42846e4b-bddc-4ec4-a4ee-d3ddf912956d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:07.448491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:07.449707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.449962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.450272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.450946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:07.452903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:07.602520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 957/1935] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:07 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:07.604390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:07.615543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c26bb-9b4f-455f-b0f2-552c18862ec7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:07.624290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:07.631015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.632510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.633168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.634854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:07.638283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:07.673480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:07.673835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 979/1936] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:07.685854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:07.690460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f270df4-068d-43d9-b38c-59ef2d1b4f1a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:07.700854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c3239-5d85-4f2d-84f2-0e3a80b7b132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:07.702603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c3239-5d85-4f2d-84f2-0e3a80b7b132 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.703071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c3239-5d85-4f2d-84f2-0e3a80b7b132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.703503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c3239-5d85-4f2d-84f2-0e3a80b7b132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.704731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c3239-5d85-4f2d-84f2-0e3a80b7b132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:07.708767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c3239-5d85-4f2d-84f2-0e3a80b7b132 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:07.723238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 958/1937] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:07.727270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:07.742333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a029e-8b8c-4919-9fa7-8c8f4f3fea54 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:07.748378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128df1bf-df17-4967-b80e-b2300053cd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:07.748834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128df1bf-df17-4967-b80e-b2300053cd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.749219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128df1bf-df17-4967-b80e-b2300053cd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.749635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128df1bf-df17-4967-b80e-b2300053cd2b admin admin] PATH_INFO: `/v3/projects/cae0808910ee419aa48babefd3cc138b/users/5ba76974954747488ca8ce1d6d98c44a/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.768758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128df1bf-df17-4967-b80e-b2300053cd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae0808910ee419aa48babefd3cc138b, user_id=5ba76974954747488ca8ce1d6d98c44a, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:07.773337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128df1bf-df17-4967-b80e-b2300053cd2b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:07.786523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6aa0-62b0-47ff-98c6-335d9e32c8a4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.786997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6aa0-62b0-47ff-98c6-335d9e32c8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.787227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6aa0-62b0-47ff-98c6-335d9e32c8a4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.789759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 959/1938] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:40:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 27 08:40:07.801854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.802266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.802654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.833267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:07.834610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:07.873684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 980/1939] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:07 2022] PUT /identity/v3/projects/cae0808910ee419aa48babefd3cc138b/users/5ba76974954747488ca8ce1d6d98c44a/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:07.884618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670f21c-8133-48a9-b197-b2e519ad3db7 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:07.884980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670f21c-8133-48a9-b197-b2e519ad3db7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:07.885342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670f21c-8133-48a9-b197-b2e519ad3db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:07.915011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b670f21c-8133-48a9-b197-b2e519ad3db7 None None] MFA Rules not processed for user `5ba76974954747488ca8ce1d6d98c44a`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:07.916312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b670f21c-8133-48a9-b197-b2e519ad3db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:07.919846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 960/1940] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:40:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:07.934475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:07.936205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:07.950492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdc5c8-da7d-486a-8b79-f67497659ef2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:07.994684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 981/1941] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:08.000372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670f21c-8133-48a9-b197-b2e519ad3db7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:08.032088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.032735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.033184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.049804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.049804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.049804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.067834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 961/1942] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:08.084216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670f21c-8133-48a9-b197-b2e519ad3db7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:08.088869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.089556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.089928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.090290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.090962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.092868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.207384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 982/1943] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:07 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:08.211720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:08.221864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8af815-382b-43ca-9110-627b6a9cd0a1 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:08.274188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:08.276291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.276900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.277337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.277621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.278346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.284125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.408782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 962/1944] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:08 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:08.410699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:08.420943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318320fe-4f80-4bd8-99ba-e07df4a989a6 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:08.425456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.426070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.426376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.426649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.427455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.429061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.439248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 983/1945] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:08.446407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:08.459009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409308b2-0ebb-469b-9036-8ec85ba3aa34 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:08.463805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.464472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.464747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.465016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] PATH_INFO: `/v3/projects/cc1e26e5821245a5997f5e7aa4ddd4f5/users/ffbdcb5f9ad940c197fabef60ed319d9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.483902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1e26e5821245a5997f5e7aa4ddd4f5, user_id=ffbdcb5f9ad940c197fabef60ed319d9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.486573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.560056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 963/1946] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:08 2022] PUT /identity/v3/projects/cc1e26e5821245a5997f5e7aa4ddd4f5/users/ffbdcb5f9ad940c197fabef60ed319d9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:08.564213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:08.576547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cabfc-f672-404c-9413-8ebfc384f17f admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:08.636760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38086f2-16fe-44c8-bf6e-2f4a49029307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.637384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38086f2-16fe-44c8-bf6e-2f4a49029307 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.637665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38086f2-16fe-44c8-bf6e-2f4a49029307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.637937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38086f2-16fe-44c8-bf6e-2f4a49029307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.638670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38086f2-16fe-44c8-bf6e-2f4a49029307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.640274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38086f2-16fe-44c8-bf6e-2f4a49029307 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.651179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 984/1947] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:08.655332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:08.666327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2b111-3cd1-4288-9b56-8a29b3fe83b4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:08.671103 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.671778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.672177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.672461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] PATH_INFO: `/v3/projects/cc1e26e5821245a5997f5e7aa4ddd4f5/users/ffbdcb5f9ad940c197fabef60ed319d9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.674496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1e26e5821245a5997f5e7aa4ddd4f5, user_id=ffbdcb5f9ad940c197fabef60ed319d9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.677773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.689234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 964/1948] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:08 2022] PUT /identity/v3/projects/cc1e26e5821245a5997f5e7aa4ddd4f5/users/ffbdcb5f9ad940c197fabef60ed319d9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:08.706838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.707273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.707594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.741154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] MFA Rules not processed for user `ffbdcb5f9ad940c197fabef60ed319d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:08.742389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:08.810274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 985/1949] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:08.821217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:08.822927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:08.830499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:08.886729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830f57-74f0-4854-a8a1-ee11bbf7db9f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:08.889805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:08.890627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:08.891245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:08.891521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:08.891791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:08.902177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:08.903740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:08.914371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 965/1950] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:09.181554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:09.268221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:09.845166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:09.886295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:09.981554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:09.983055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:09.994953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5ae28-599c-4bc0-826a-8a3e673ce683 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:09.998446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072ee48-1857-4b7b-aae2-c9560cfe19d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:10.002854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ee48-1857-4b7b-aae2-c9560cfe19d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:10.002854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ee48-1857-4b7b-aae2-c9560cfe19d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:10.002854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ee48-1857-4b7b-aae2-c9560cfe19d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:10.059776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072ee48-1857-4b7b-aae2-c9560cfe19d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:10.061224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072ee48-1857-4b7b-aae2-c9560cfe19d7 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:10.070463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 986/1951] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:10.096649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:10.177475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:10.234180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:10.730418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:10.795606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.051110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.085019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.171028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.249697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.571452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.639437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.685080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.731152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:11.965963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:12.078221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:12.683961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:13.280009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:15.358524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:15.359970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:15.498632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aca5d-1f56-41d1-ac32-0664ff0c4979 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:15.505617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:15.506441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:15.506796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:15.507168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] PATH_INFO: `/v3/users/fe245f6195114b8ab1ec98cd683876f2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:15.508524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe245f6195114b8ab1ec98cd683876f2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:15.511072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:15.800417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:15.803831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe245f6195114b8ab1ec98cd683876f2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:15.816148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe245f6195114b8ab1ec98cd683876f2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:15.879459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe245f6195114b8ab1ec98cd683876f2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:15.886974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe245f6195114b8ab1ec98cd683876f2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:15.887638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe245f6195114b8ab1ec98cd683876f2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:15.893443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 966/1952] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:15 2022] DELETE /identity/v3/users/fe245f6195114b8ab1ec98cd683876f2 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:15.898398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:15.898938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:15.899051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:15.920778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:15.922099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:15.936290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:15.984761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 987/1953] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:15.994730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:15.996159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:16.064328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aff592-8596-4e96-825d-ac00d30ebbff admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:16.068683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:16.069395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:16.069573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:16.069844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:16.086170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:16.087772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:16.097287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 967/1954] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:16.555387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:16.556936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:16.573652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e5d56-ec31-448a-b5a0-704ca1574ccd None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:16.578820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:16.579581 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:16.579956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:16.580406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] PATH_INFO: `/v3/projects/2e106d38d8b84299808e60a1deb59480` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:16.582054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e106d38d8b84299808e60a1deb59480)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:16.585621 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:16.705383 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e106d38d8b84299808e60a1deb59480', 'request_id': 'req-fe975f27-52a5-446c-b143-6480c285f261'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:16.713743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e106d38d8b84299808e60a1deb59480', 'request_id': 'req-fe975f27-52a5-446c-b143-6480c285f261'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:16.778525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e106d38d8b84299808e60a1deb59480', 'request_id': 'req-fe975f27-52a5-446c-b143-6480c285f261'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:16.806463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] The token cache is being invalidate because project 2e106d38d8b84299808e60a1deb59480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:16.807961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:16.812475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:16.815030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 988/1955] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:16 2022] DELETE /identity/v3/projects/2e106d38d8b84299808e60a1deb59480 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:16.819548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:16.835410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e9dad-10ec-4de3-8bb9-8ce2717cd1e4 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:16.904912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:16.905596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:16.905877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:16.906178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] PATH_INFO: `/v3/users/8902e533927b4b598cb5a717d5156173` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:16.907514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8902e533927b4b598cb5a717d5156173)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:16.910363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:17.169435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8902e533927b4b598cb5a717d5156173'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:17.183114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8902e533927b4b598cb5a717d5156173'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:17.184997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:17.267648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8902e533927b4b598cb5a717d5156173'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:17.275734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8902e533927b4b598cb5a717d5156173'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:17.276398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8902e533927b4b598cb5a717d5156173'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:17.282497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 968/1956] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:16 2022] DELETE /identity/v3/users/8902e533927b4b598cb5a717d5156173 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:17.692465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:17.693894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:17.708216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe975f27-52a5-446c-b143-6480c285f261 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:17.783455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:17.784193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:17.784558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:17.784976 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] PATH_INFO: `/v3/projects/185895c0893c4617a435e843e971582f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:17.787353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=185895c0893c4617a435e843e971582f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:17.789344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:17.815769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:17.817697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:17.835842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4dcbb-df52-417a-a85e-9a54203afa1d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:17.840635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:17.841340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:17.841621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:17.841894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] PATH_INFO: `/v3/users/153baa095b9a40028f21daa9d8d916b8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:17.843579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=153baa095b9a40028f21daa9d8d916b8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:17.845539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:17.893270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:17.897261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185895c0893c4617a435e843e971582f', 'request_id': 'req-105105e5-b3be-47a0-b770-1682a8a73ce3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:17.914180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185895c0893c4617a435e843e971582f', 'request_id': 'req-105105e5-b3be-47a0-b770-1682a8a73ce3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:17.988639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185895c0893c4617a435e843e971582f', 'request_id': 'req-105105e5-b3be-47a0-b770-1682a8a73ce3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.014494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] The token cache is being invalidate because project 185895c0893c4617a435e843e971582f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:18.014494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.014494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.024328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 989/1957] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:17 2022] DELETE /identity/v3/projects/185895c0893c4617a435e843e971582f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:18.025378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:18.027566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153baa095b9a40028f21daa9d8d916b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.029595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:18.078173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153baa095b9a40028f21daa9d8d916b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.121404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105105e5-b3be-47a0-b770-1682a8a73ce3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:18.127234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd5edc-a40e-44ca-a734-a153b7c788f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:18.127595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd5edc-a40e-44ca-a734-a153b7c788f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:18.127949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd5edc-a40e-44ca-a734-a153b7c788f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:18.128353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd5edc-a40e-44ca-a734-a153b7c788f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:18.130556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-23cd5edc-a40e-44ca-a734-a153b7c788f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:18.187379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153baa095b9a40028f21daa9d8d916b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.195665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153baa095b9a40028f21daa9d8d916b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.196640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153baa095b9a40028f21daa9d8d916b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.210299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd5edc-a40e-44ca-a734-a153b7c788f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:18.212586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd5edc-a40e-44ca-a734-a153b7c788f8 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:18.213738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 969/1958] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:17 2022] DELETE /identity/v3/users/153baa095b9a40028f21daa9d8d916b8 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:18.214785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:18.219138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:18.227728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 990/1959] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:18.229481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:18.229847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:18.230250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:18.258262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:18.260023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:18.352031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae59a95-ad9c-4119-8947-efbcab6516e5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:18.356959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:18.357610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:18.357884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:18.358183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] PATH_INFO: `/v3/users/f8ee3f8d323a4d9db6e19e6cd8ab2760` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:18.359500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ee3f8d323a4d9db6e19e6cd8ab2760)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:18.361780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:18.369674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 991/1960] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:18.381257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:18.386025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:18.462850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e6cf-e6ac-4e17-b4a4-b1e8a0821645 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:18.466184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:18.466919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:18.469590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:18.469590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:18.479429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:18.481379 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:18.502567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 992/1961] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:18.518171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:18.659265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ee3f8d323a4d9db6e19e6cd8ab2760'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.666910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=2,keystone:DELETE=19,keystone:SELECT=79 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:18.672171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ee3f8d323a4d9db6e19e6cd8ab2760'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.736814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ee3f8d323a4d9db6e19e6cd8ab2760'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.744582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ee3f8d323a4d9db6e19e6cd8ab2760'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.745297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ee3f8d323a4d9db6e19e6cd8ab2760'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:18.752789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 970/1962] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:18 2022] DELETE /identity/v3/users/f8ee3f8d323a4d9db6e19e6cd8ab2760 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:18.978315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:18.979771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:19.008365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df10e3-a04b-4075-9405-d077709c6226 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:19.165967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:19.167066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.168214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.168492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] PATH_INFO: `/v3/projects/66825c1377de4136bda787d5d4e78c23` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.169751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66825c1377de4136bda787d5d4e78c23)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:19.171946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:19.349195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66825c1377de4136bda787d5d4e78c23', 'request_id': 'req-1a095576-5e01-4bad-b63d-293de338d986'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.355895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66825c1377de4136bda787d5d4e78c23', 'request_id': 'req-1a095576-5e01-4bad-b63d-293de338d986'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.425749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:19.428912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:19.443683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98acc-bba6-4eb0-a29d-181ab061864a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:19.473844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66825c1377de4136bda787d5d4e78c23', 'request_id': 'req-1a095576-5e01-4bad-b63d-293de338d986'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.490731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] The token cache is being invalidate because project 66825c1377de4136bda787d5d4e78c23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:19.491264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.492219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1a095576-5e01-4bad-b63d-293de338d986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.495077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 993/1963] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:18 2022] DELETE /identity/v3/projects/66825c1377de4136bda787d5d4e78c23 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:19.499940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299dc12-88ff-4778-93ba-5ccd669a3fbc None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.500413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299dc12-88ff-4778-93ba-5ccd669a3fbc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.500969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299dc12-88ff-4778-93ba-5ccd669a3fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.570963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6299dc12-88ff-4778-93ba-5ccd669a3fbc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:19.572213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6299dc12-88ff-4778-93ba-5ccd669a3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:19.575415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:19.576383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.576849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.577352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] PATH_INFO: `/v3/projects/aac629981b764f90884a041d600d2405` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.579018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac629981b764f90884a041d600d2405)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:19.581857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:19.593403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 994/1964] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:19.601939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b247ba-f3bd-476c-81f6-72ebe1120e46 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.602500 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b247ba-f3bd-476c-81f6-72ebe1120e46 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.602931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b247ba-f3bd-476c-81f6-72ebe1120e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.628267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-94b247ba-f3bd-476c-81f6-72ebe1120e46 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:19.629648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-94b247ba-f3bd-476c-81f6-72ebe1120e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:19.642200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 995/1965] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:19.645727 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b247ba-f3bd-476c-81f6-72ebe1120e46 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:19.655939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b247ba-f3bd-476c-81f6-72ebe1120e46 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:19.661346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:19.661992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.662310 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.662584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.663282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:19.663586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:19.665184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:19.678212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 996/1966] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:19.682982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:19.694769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45be3b-2aac-42b0-9ff9-c3e1d8ad46e2 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:19.697794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac629981b764f90884a041d600d2405', 'request_id': 'req-12f1b308-73f5-472b-a5ba-b852cceb199f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.711503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac629981b764f90884a041d600d2405', 'request_id': 'req-12f1b308-73f5-472b-a5ba-b852cceb199f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.722282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac629981b764f90884a041d600d2405', 'request_id': 'req-12f1b308-73f5-472b-a5ba-b852cceb199f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.739288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:19.765955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:19.766696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.766979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.767257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.767961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:19.770942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:19.808866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] The token cache is being invalidate because project aac629981b764f90884a041d600d2405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:19.809317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.810052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12f1b308-73f5-472b-a5ba-b852cceb199f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:19.812547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 971/1967] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:19 2022] DELETE /identity/v3/projects/aac629981b764f90884a041d600d2405 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:19.819353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01e8c4-4524-4363-b6e1-78a9d8b1c174 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.819642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01e8c4-4524-4363-b6e1-78a9d8b1c174 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.819953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01e8c4-4524-4363-b6e1-78a9d8b1c174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.841235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6a01e8c4-4524-4363-b6e1-78a9d8b1c174 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:19.846278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6a01e8c4-4524-4363-b6e1-78a9d8b1c174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:19.864132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 972/1968] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:19.864667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.865768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.866440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.888401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:19.890073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:19.902188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 973/1969] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:19.902354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 997/1970] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:19 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:19.903753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:19.903877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:19.905683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:19.918957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18983d-ae2b-43c1-9c40-9f14e83227ce None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:19.980698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:19.992219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:19.992942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:19.993294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:19.993638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:19.994482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:19.998231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.002784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d81d44-c1a6-431c-98e0-eaa6d66e29ab admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.007918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.008881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.009227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.009705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.010510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:20.011082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.013571 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.029268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 998/1971] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:20.034902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.062416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49975b36-9826-49d7-b9aa-425879718377 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.069485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.070459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.070740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.071012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.072343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.074924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.117156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 974/1972] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:19 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:20.118319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.145930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccc3f8-420d-495b-821a-b81c88917fef admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.145930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.145930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.145930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.146364 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.146364 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.151209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.176743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 975/1973] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:20.180012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.199475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c6826-38cb-456c-b475-9ce66d893560 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.205048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.205779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.206420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.206805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 999/1974] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:20 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:20.207788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.208855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.210687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.218476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.226254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 976/1975] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:20.230685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.249201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c5219-f563-4a10-8273-d9d2116c99d7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.337313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9422ac-4904-46f4-af52-8f338c255bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.339950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9422ac-4904-46f4-af52-8f338c255bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.340466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9422ac-4904-46f4-af52-8f338c255bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.341213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9422ac-4904-46f4-af52-8f338c255bc9 admin admin] PATH_INFO: `/v3/projects/c22468b313144406a260b57953b0ecca/users/6ac2338d2ef740f695a280bdfdeac5e7/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.358058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a636f5-aafb-46c3-bcad-0bd62fe37ab0 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.361836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9422ac-4904-46f4-af52-8f338c255bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22468b313144406a260b57953b0ecca, user_id=6ac2338d2ef740f695a280bdfdeac5e7, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.365982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9422ac-4904-46f4-af52-8f338c255bc9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.372175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.372875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.373240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.375446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.375446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.377294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.448766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 977/1976] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:20 2022] PUT /identity/v3/projects/c22468b313144406a260b57953b0ecca/users/6ac2338d2ef740f695a280bdfdeac5e7/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:20.459666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971c23-e648-4b2f-b218-de2b896875d2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.460046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971c23-e648-4b2f-b218-de2b896875d2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.460412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971c23-e648-4b2f-b218-de2b896875d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.492743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4a971c23-e648-4b2f-b218-de2b896875d2 None None] MFA Rules not processed for user `6ac2338d2ef740f695a280bdfdeac5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:20.493974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4a971c23-e648-4b2f-b218-de2b896875d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:20.508479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1000/1977] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:20 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:20.513746 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.524724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ff318-4cd2-46f9-bc8d-e75e5b4c7f2b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.591850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 978/1978] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:20.596291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971c23-e648-4b2f-b218-de2b896875d2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.650700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.651419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.651902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.652309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.653253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.655221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.684143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1001/1979] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:20.685270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.696489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94ac3d-0ffc-4d18-986f-654c61a4abeb admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.697156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971c23-e648-4b2f-b218-de2b896875d2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.701262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.701888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.702189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.702668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.703374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.703846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.704577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.704954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.705360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.705930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.707988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.710506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.718895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1002/1980] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:20.723364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.736090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6afdc-a9a8-4a57-80a8-c4dcc3732a84 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.741340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31255ed-1a68-439b-86e3-4c15b6aa3479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:20.742027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31255ed-1a68-439b-86e3-4c15b6aa3479 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.742338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31255ed-1a68-439b-86e3-4c15b6aa3479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.742614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31255ed-1a68-439b-86e3-4c15b6aa3479 admin admin] PATH_INFO: `/v3/projects/406cb19b25a24613bb0149d9386a67ff/users/795fb1879bb041b48a58c602dcf36e44/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.761034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31255ed-1a68-439b-86e3-4c15b6aa3479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406cb19b25a24613bb0149d9386a67ff, user_id=795fb1879bb041b48a58c602dcf36e44, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:20.763965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31255ed-1a68-439b-86e3-4c15b6aa3479 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:20.817530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 979/1981] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:20 2022] POST /identity/v3/projects => generated 373 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:20.821650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:20.887678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1003/1982] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:20 2022] PUT /identity/v3/projects/406cb19b25a24613bb0149d9386a67ff/users/795fb1879bb041b48a58c602dcf36e44/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:20.889249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:20.889776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:20.889776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:20.919840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbaaba-bdc7-4450-8879-c58f22fd15b5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:20.929285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] MFA Rules not processed for user `795fb1879bb041b48a58c602dcf36e44`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:20.930624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.040955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.041508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.041904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.042290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.043462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.046224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.061544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1004/1983] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:21.073786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.075265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.079862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:21.156753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 980/1984] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:20 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:21.159520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1998dc0-c5b4-48b3-9340-9f7f10ab6304 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:21.166450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.167109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.167506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.167691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.167942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.194855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.197815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.201292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe283-8c23-4b9d-bc9d-afeba012b94f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:21.212570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.217655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.217655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.217655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.217655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.217655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.220099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1005/1985] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:21.229361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 981/1986] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:21.236086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.247271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.256229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67452799-f3dc-40c3-98a1-c976de56a710 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:21.261065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.261749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.262033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.262342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] PATH_INFO: `/v3/projects/e9ca2ad15f8f44709e621d245a765bde/users/dc848ef4fb58440da832b9c5deafd4a9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.271200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970d0ca-0228-44fa-bc10-af27ec6c62c4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:21.276580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.277434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.277777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.278423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] PATH_INFO: `/v3/users/795fb1879bb041b48a58c602dcf36e44` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.279619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] RBAC: Authorizing `identity:delete_user(user_id=795fb1879bb041b48a58c602dcf36e44)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.280982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ca2ad15f8f44709e621d245a765bde, user_id=dc848ef4fb58440da832b9c5deafd4a9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.281530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.284886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.354219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:21.388313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1006/1987] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:21 2022] PUT /identity/v3/projects/e9ca2ad15f8f44709e621d245a765bde/users/dc848ef4fb58440da832b9c5deafd4a9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:21.389207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.391019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.460491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb5844-e223-413d-9d47-171a5a8d1c23 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:21.466954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.467404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795fb1879bb041b48a58c602dcf36e44'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.467862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.467862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.468118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] PATH_INFO: `/v3/users/d74fceb9edc945cda61f8a6883ba4f74` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.469730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74fceb9edc945cda61f8a6883ba4f74)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.471699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.482242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795fb1879bb041b48a58c602dcf36e44'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.536436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795fb1879bb041b48a58c602dcf36e44'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.546938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795fb1879bb041b48a58c602dcf36e44'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.547718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795fb1879bb041b48a58c602dcf36e44'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.563770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 982/1988] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:21 2022] DELETE /identity/v3/users/795fb1879bb041b48a58c602dcf36e44 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:21.564692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.574421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.655590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74fceb9edc945cda61f8a6883ba4f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.683096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74fceb9edc945cda61f8a6883ba4f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.713834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cedb7d-ca84-4cd6-85d4-e4785be9747f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:21.721032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b13318-b9f8-4fb9-9a0d-61465d5e3c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.721718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13318-b9f8-4fb9-9a0d-61465d5e3c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.722006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13318-b9f8-4fb9-9a0d-61465d5e3c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.722316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13318-b9f8-4fb9-9a0d-61465d5e3c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.723592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b13318-b9f8-4fb9-9a0d-61465d5e3c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.725098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b13318-b9f8-4fb9-9a0d-61465d5e3c8e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.736741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:21.753342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 983/1989] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:21.755811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c5d2d-c722-4332-9d31-5cd6919a6062 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.758474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c5d2d-c722-4332-9d31-5cd6919a6062 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.758474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c5d2d-c722-4332-9d31-5cd6919a6062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.776695 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74fceb9edc945cda61f8a6883ba4f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.777546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74fceb9edc945cda61f8a6883ba4f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.778905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74fceb9edc945cda61f8a6883ba4f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:21.798133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1007/1990] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:21 2022] DELETE /identity/v3/users/d74fceb9edc945cda61f8a6883ba4f74 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:21.802671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f58c5d2d-c722-4332-9d31-5cd6919a6062 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:21.803152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.803963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f58c5d2d-c722-4332-9d31-5cd6919a6062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.805700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.857527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:21.895086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 984/1991] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:21.896412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.896704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.896993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.909985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8791c72-c8af-433b-ab67-e6ab8dc2e2d2 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:21.920128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:21.920925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:21.923647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:21.923647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:21.923647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] PATH_INFO: `/v3/projects/e9ca2ad15f8f44709e621d245a765bde/users/dc848ef4fb58440da832b9c5deafd4a9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:21.923958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.925100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ca2ad15f8f44709e621d245a765bde, user_id=dc848ef4fb58440da832b9c5deafd4a9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:21.928548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:21.932923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 985/1992] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:21.933742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.935035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:21.953600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1008/1993] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:21 2022] PUT /identity/v3/projects/e9ca2ad15f8f44709e621d245a765bde/users/dc848ef4fb58440da832b9c5deafd4a9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:21.964717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:21.964717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:22.012606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391fb0-9870-4e22-b788-765b42d65d68 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:22.047777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bb230-8d1a-4e2c-8c01-d8f838628d54 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:22.052389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4fa78-186a-42e6-9899-9fb5274357db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:22.053145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4fa78-186a-42e6-9899-9fb5274357db service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:22.053525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4fa78-186a-42e6-9899-9fb5274357db service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:22.053906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4fa78-186a-42e6-9899-9fb5274357db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:22.088478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:22.089182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:22.090045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:22.091216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:22.141662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4fa78-186a-42e6-9899-9fb5274357db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:22.147169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4fa78-186a-42e6-9899-9fb5274357db service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:22.161023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1009/1994] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:22.162648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:22.163006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:22.163283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:22.217485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] MFA Rules not processed for user `dc848ef4fb58440da832b9c5deafd4a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:22.218572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:22.218759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:22.232341 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:22.253712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 986/1995] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:22.332889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1010/1996] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:22.346345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:22.349435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:22.422120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:22.425742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43b0a-ff59-4159-9922-7bc1b090c874 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:22.430500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:22.431140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:22.431430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:22.431707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:22.433899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:22.513603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:22.514815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:22.531059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 987/1997] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:22.720550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:23.045037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:23.046644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:23.070113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bc42f-e631-4abc-8483-3084cbc91711 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:23.095629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:23.096670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:23.098729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:23.100760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] PATH_INFO: `/v3/projects/13e24414aaae47afaadde1a97ff9ab76` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:23.114643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=13e24414aaae47afaadde1a97ff9ab76)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:23.128218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:23.324215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:23.339815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:23.341719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:23.364828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640957a6-3bc4-4042-a96e-ae725a0d2684 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:23.369990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13e24414aaae47afaadde1a97ff9ab76', 'request_id': 'req-eea8c118-0df1-4ba8-8012-5e238ad5e14e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.397330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13e24414aaae47afaadde1a97ff9ab76', 'request_id': 'req-eea8c118-0df1-4ba8-8012-5e238ad5e14e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.495839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13e24414aaae47afaadde1a97ff9ab76', 'request_id': 'req-eea8c118-0df1-4ba8-8012-5e238ad5e14e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.515046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:23.516403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:23.516833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:23.517254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] PATH_INFO: `/v3/projects/406cb19b25a24613bb0149d9386a67ff` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:23.518065 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] The token cache is being invalidate because project 13e24414aaae47afaadde1a97ff9ab76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:23.519081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=406cb19b25a24613bb0149d9386a67ff)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:23.519534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.522768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:23.525255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.528522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1011/1998] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:23 2022] DELETE /identity/v3/projects/13e24414aaae47afaadde1a97ff9ab76 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:23.536485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:23.540622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:23.606655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8c118-0df1-4ba8-8012-5e238ad5e14e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:23.620905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:23.622705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:23.623364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:23.624475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] PATH_INFO: `/v3/users/79f168e4d95149519e1408612dd40e1f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:23.626785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f168e4d95149519e1408612dd40e1f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:23.629864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:23.659679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406cb19b25a24613bb0149d9386a67ff', 'request_id': 'req-ff5a627a-9495-46fe-91b7-c937ac11c2fe'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.676962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406cb19b25a24613bb0149d9386a67ff', 'request_id': 'req-ff5a627a-9495-46fe-91b7-c937ac11c2fe'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.690587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406cb19b25a24613bb0149d9386a67ff', 'request_id': 'req-ff5a627a-9495-46fe-91b7-c937ac11c2fe'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.694198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:23.799418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] The token cache is being invalidate because project 406cb19b25a24613bb0149d9386a67ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:23.799814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.800483 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.803074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 988/1999] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:23 2022] DELETE /identity/v3/projects/406cb19b25a24613bb0149d9386a67ff => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:23.803778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:23.805134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:23.873094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f168e4d95149519e1408612dd40e1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.885842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a627a-9495-46fe-91b7-c937ac11c2fe admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:23.891792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:23.894486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:23.894486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:23.894486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] PATH_INFO: `/v3/users/7d388aeb377d40cdad0c4af57e7bd122` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:23.895338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d388aeb377d40cdad0c4af57e7bd122)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:23.896755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:23.909782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f168e4d95149519e1408612dd40e1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.970536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f168e4d95149519e1408612dd40e1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.971407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f168e4d95149519e1408612dd40e1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.971956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ef4a5aa8-0bc3-418f-be8f-a12528a8d897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f168e4d95149519e1408612dd40e1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:23.986991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1012/2000] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:23 2022] DELETE /identity/v3/users/79f168e4d95149519e1408612dd40e1f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:23.989206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c7d6f-3af5-48ca-bc37-c0b9a0bc98ad None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:23.989929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c7d6f-3af5-48ca-bc37-c0b9a0bc98ad None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:23.990524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c7d6f-3af5-48ca-bc37-c0b9a0bc98ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:24.013364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-cd1c7d6f-3af5-48ca-bc37-c0b9a0bc98ad None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:24.014645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cd1c7d6f-3af5-48ca-bc37-c0b9a0bc98ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:24.104936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1013/2001] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:24.106429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe373a30-3c2e-428e-963c-7750a2c9f9ca None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:24.106717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe373a30-3c2e-428e-963c-7750a2c9f9ca None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:24.107047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe373a30-3c2e-428e-963c-7750a2c9f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:24.131463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fe373a30-3c2e-428e-963c-7750a2c9f9ca None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:24.133209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fe373a30-3c2e-428e-963c-7750a2c9f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:24.137941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d388aeb377d40cdad0c4af57e7bd122'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:24.144669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1014/2002] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:24.146350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:24.146638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:24.146909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:24.150422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:24.163828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d388aeb377d40cdad0c4af57e7bd122'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:24.170701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:24.176109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:24.183850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1015/2003] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:24.184786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:24.187043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:24.256066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d388aeb377d40cdad0c4af57e7bd122'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:24.268447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d03a-dd02-4523-996c-6e30972719f1 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:24.270948 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d388aeb377d40cdad0c4af57e7bd122'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:24.272666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d388aeb377d40cdad0c4af57e7bd122'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:24.273299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:24.274000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:24.274702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:24.274839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:24.277352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:24.295970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 989/2004] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:23 2022] DELETE /identity/v3/users/7d388aeb377d40cdad0c4af57e7bd122 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:24.299666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:24.304549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:24.328140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:24.334391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:24.335267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:24.359181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1016/2005] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:24.411523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8625772-566d-484f-9386-853d14396db3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:24.417037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:24.417897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:24.418321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:24.419578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:24.420399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:24.420860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:24.422678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:24.457174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 990/2006] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:24.458332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:24.502684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d23b3-da36-489b-8347-64168214c4b5 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:24.528159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:24.532536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:24.532987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:24.533407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:24.539309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:24.540613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:24.716697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1017/2007] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:24 2022] POST /identity/v3/projects => generated 369 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:24.720687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:24.737689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000b312-16b8-41cb-bf6e-c7de7baea7a0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:24.819119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:24.820403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:24.820840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:24.821482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:24.822518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:24.825345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:24.968249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 991/2008] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:24 2022] POST /identity/v3/users => generated 285 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:24.971753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:24.986521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf763fb-52b2-4500-b952-0c551769f811 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:24.999587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:25.000651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.000651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.000853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.001516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:25.005188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:25.029539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1018/2009] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:25.033643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:25.048237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afa7d3-6e6d-4690-adb9-7705bf12ef93 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:25.060481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:25.061122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.061405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.061683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.062501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:25.064669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:25.083178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 992/2010] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:25.086990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:25.106319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e976232-d06e-47b1-a876-7f37c41b1325 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:25.111844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:25.112537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.113747 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.114440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] PATH_INFO: `/v3/projects/3bc58d1e7ca44ed0b776999dd09beb0a/users/383f0bd30d0249938d35fa622177c457/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.141960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc58d1e7ca44ed0b776999dd09beb0a, user_id=383f0bd30d0249938d35fa622177c457, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:25.146054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:25.237105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:25.240779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:25.288333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1019/2011] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:25 2022] PUT /identity/v3/projects/3bc58d1e7ca44ed0b776999dd09beb0a/users/383f0bd30d0249938d35fa622177c457/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:25.289298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd455c-d985-41f0-87a3-d5c1e2f15143 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:25.289834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:25.291936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:25.376830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f25ff-6637-48fd-8e74-2f79820034ff admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:25.381828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:25.382473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:25.383207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.383564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.383942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] PATH_INFO: `/v3/projects/1963465f3fe8443a90998e8a762e4be0` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.385505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1963465f3fe8443a90998e8a762e4be0)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:25.393920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:25.419268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:25.419913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.420202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.420472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] PATH_INFO: `/v3/projects/95e197182adb4c9cb08c265c140e6fdc` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.422938 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e197182adb4c9cb08c265c140e6fdc)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:25.423679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:25.537569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1963465f3fe8443a90998e8a762e4be0', 'request_id': 'req-3ace0223-513e-432c-ba2b-c28df9b282a4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.544232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e197182adb4c9cb08c265c140e6fdc', 'request_id': 'req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.545647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1963465f3fe8443a90998e8a762e4be0', 'request_id': 'req-3ace0223-513e-432c-ba2b-c28df9b282a4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.556595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e197182adb4c9cb08c265c140e6fdc', 'request_id': 'req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.564086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e197182adb4c9cb08c265c140e6fdc', 'request_id': 'req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.647807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1963465f3fe8443a90998e8a762e4be0', 'request_id': 'req-3ace0223-513e-432c-ba2b-c28df9b282a4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.663003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:25.671573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] The token cache is being invalidate because project 1963465f3fe8443a90998e8a762e4be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:25.671869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.677055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3ace0223-513e-432c-ba2b-c28df9b282a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.679846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1020/2012] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:25 2022] DELETE /identity/v3/projects/1963465f3fe8443a90998e8a762e4be0 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:25.681246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e660ae-5779-49e8-838a-259d255134db None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.681546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e660ae-5779-49e8-838a-259d255134db None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.681810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e660ae-5779-49e8-838a-259d255134db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.713305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] The token cache is being invalidate because project 95e197182adb4c9cb08c265c140e6fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:25.713305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.713761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:25.715568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 993/2013] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:25 2022] DELETE /identity/v3/projects/95e197182adb4c9cb08c265c140e6fdc => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:25.716569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:25.721085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:25.749898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-70e660ae-5779-49e8-838a-259d255134db None None] MFA Rules not processed for user `383f0bd30d0249938d35fa622177c457`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:25.751143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-70e660ae-5779-49e8-838a-259d255134db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:25.823816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64938-faf1-4ff5-8eab-277b2dbf52ed admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:25.832672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:25.833773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.834751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.835201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] PATH_INFO: `/v3/users/5ba76974954747488ca8ce1d6d98c44a` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.837061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba76974954747488ca8ce1d6d98c44a)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:25.840015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:25.896226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1021/2014] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:25.897694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:25.898373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:25.898730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:25.936355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:25.937610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:25.946877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1022/2015] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:25.955641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:25.958284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:26.038710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddf1ec-6187-4474-86d0-b13a052f9850 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:26.042406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9587749-f3e1-45b4-a67f-3ee65b3dce2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:26.043029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9587749-f3e1-45b4-a67f-3ee65b3dce2b service glance] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.043439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9587749-f3e1-45b4-a67f-3ee65b3dce2b service glance] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.043606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9587749-f3e1-45b4-a67f-3ee65b3dce2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.053251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9587749-f3e1-45b4-a67f-3ee65b3dce2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:26.054636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9587749-f3e1-45b4-a67f-3ee65b3dce2b service glance] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:26.063782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1023/2016] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:26.065027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef1100-e1b2-4e5e-96cf-eef6b0b788d5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.065316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef1100-e1b2-4e5e-96cf-eef6b0b788d5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.065593 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef1100-e1b2-4e5e-96cf-eef6b0b788d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.086675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f0ef1100-e1b2-4e5e-96cf-eef6b0b788d5 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:26.087901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f0ef1100-e1b2-4e5e-96cf-eef6b0b788d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:26.099220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1024/2017] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:26.101730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef1100-e1b2-4e5e-96cf-eef6b0b788d5 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:26.110563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef1100-e1b2-4e5e-96cf-eef6b0b788d5 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:26.116117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:26.116724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.117005 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.117282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.117975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:26.118790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:26.120303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:26.132775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1025/2018] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:40:26.137027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:26.147112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7da8c9-f644-44de-9812-7a9aba919ba1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:26.152109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:26.152719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.153002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.153273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.154057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:26.156016 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:26.209756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba76974954747488ca8ce1d6d98c44a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:26.218577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:26.224302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba76974954747488ca8ce1d6d98c44a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:26.261910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1026/2019] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:26 2022] POST /identity/v3/projects => generated 409 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:26.262159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:26.309179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:26.336159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba76974954747488ca8ce1d6d98c44a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:26.345832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba76974954747488ca8ce1d6d98c44a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:26.347813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba76974954747488ca8ce1d6d98c44a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:26.358951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 994/2020] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:25 2022] DELETE /identity/v3/users/5ba76974954747488ca8ce1d6d98c44a => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:26.361957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe762-8797-4320-99b4-c76d9805be73 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:26.459092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:26.459831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.463620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.464009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.464800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:26.467099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:26.586702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1027/2021] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:26 2022] POST /identity/v3/users => generated 305 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:26.588346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:26.591273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:26.607987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be5ffd-2cbc-4b30-a9e3-37da4243ff6b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:26.613292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:26.614892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.614892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.614892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.616064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:26.618027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:26.635250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 995/2022] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:26.636896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:26.648536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729deeb6-940d-4e9c-aec1-b981f48c7455 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:26.654734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c769da-fce2-449f-a997-6ae73aea89cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:26.655553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c769da-fce2-449f-a997-6ae73aea89cb admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.655857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c769da-fce2-449f-a997-6ae73aea89cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.656491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c769da-fce2-449f-a997-6ae73aea89cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.657307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c769da-fce2-449f-a997-6ae73aea89cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:26.659491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c769da-fce2-449f-a997-6ae73aea89cb admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:26.680725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1028/2023] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:26.680883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:26.703617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31c18f-e87b-46c8-8e83-95acc5f6004e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:26.711783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:26.711783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.724040 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.725826 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] PATH_INFO: `/v3/projects/41384c34da9f46e59a12bc80d794ac03/users/712acc57d05b432fa9f5d7a7c09dec39/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.755976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41384c34da9f46e59a12bc80d794ac03, user_id=712acc57d05b432fa9f5d7a7c09dec39, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:26.760395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:26.842056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 996/2024] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:26 2022] PUT /identity/v3/projects/41384c34da9f46e59a12bc80d794ac03/users/712acc57d05b432fa9f5d7a7c09dec39/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:26.853178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:26.853543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:26.855144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:26.889374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] MFA Rules not processed for user `712acc57d05b432fa9f5d7a7c09dec39`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:26.890957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:26.989743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1029/2025] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:26.994199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:27.073832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510b5c8-b423-4fa4-9c7a-24b46d1797e6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:27.078898 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:27.079530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:27.079815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:27.080093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:27.082345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:27.084244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:27.136213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:27.138708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:27.188561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421ce8a-2507-4e06-b262-e601baa0ec52 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:27.196417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:27.199275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:27.199783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:27.202869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] PATH_INFO: `/v3/projects/cae0808910ee419aa48babefd3cc138b` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:27.208757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae0808910ee419aa48babefd3cc138b)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:27.213013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:27.221842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 997/2026] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:26 2022] POST /identity/v3/projects => generated 409 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:27.224863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:27.341619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:27.352758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd155bc1-e13a-4073-9d75-f251a19a2a84 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:27.356925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae0808910ee419aa48babefd3cc138b', 'request_id': 'req-2e1e3199-fa0e-446e-860f-ab500602f8b4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:27.362957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae0808910ee419aa48babefd3cc138b', 'request_id': 'req-2e1e3199-fa0e-446e-860f-ab500602f8b4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:27.389258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:27.417658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:27.418321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:27.418819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:27.418932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:27.419568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:27.421490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:27.453450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae0808910ee419aa48babefd3cc138b', 'request_id': 'req-2e1e3199-fa0e-446e-860f-ab500602f8b4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:27.467881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] The token cache is being invalidate because project cae0808910ee419aa48babefd3cc138b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:27.468491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:27.469063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:27.470936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1030/2027] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:27 2022] DELETE /identity/v3/projects/cae0808910ee419aa48babefd3cc138b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:27.472608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:27.475712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:27.489914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e3199-fa0e-446e-860f-ab500602f8b4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:27.494802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:27.495511 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:27.495866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:27.496240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] PATH_INFO: `/v3/users/6ac2338d2ef740f695a280bdfdeac5e7` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:27.497485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac2338d2ef740f695a280bdfdeac5e7)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:27.499624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:27.539684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 998/2028] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:27 2022] POST /identity/v3/users => generated 311 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:27.539684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:27.541424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:27.558379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd93cd5-465b-4ced-8051-18c4ffb81267 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:27.564269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:27.565005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:27.565365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:27.565702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] PATH_INFO: `/v3/users/ffbdcb5f9ad940c197fabef60ed319d9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:27.571318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbdcb5f9ad940c197fabef60ed319d9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:27.573394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:28.103235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac2338d2ef740f695a280bdfdeac5e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.116161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac2338d2ef740f695a280bdfdeac5e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.590003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbdcb5f9ad940c197fabef60ed319d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.601539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbdcb5f9ad940c197fabef60ed319d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.604055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac2338d2ef740f695a280bdfdeac5e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.608242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac2338d2ef740f695a280bdfdeac5e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.608242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac2338d2ef740f695a280bdfdeac5e7'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.618282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1031/2029] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:27 2022] DELETE /identity/v3/users/6ac2338d2ef740f695a280bdfdeac5e7 => generated 0 bytes in 1147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:28.619106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:28.620540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:28.685995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b5dd4-2270-4591-9dcb-1bc5ddff268d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:28.690588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ef6c9-9a08-491b-8fe2-320b4dbb75b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:28.691911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ef6c9-9a08-491b-8fe2-320b4dbb75b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:28.692218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ef6c9-9a08-491b-8fe2-320b4dbb75b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:28.692496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ef6c9-9a08-491b-8fe2-320b4dbb75b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:28.693184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ef6c9-9a08-491b-8fe2-320b4dbb75b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:28.694798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ef6c9-9a08-491b-8fe2-320b4dbb75b7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:28.704706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1032/2030] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:28.709116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16555d29-8746-4821-8d21-82690afd8e4f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:28.709116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16555d29-8746-4821-8d21-82690afd8e4f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:28.709116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16555d29-8746-4821-8d21-82690afd8e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:28.725862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-16555d29-8746-4821-8d21-82690afd8e4f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:28.727098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-16555d29-8746-4821-8d21-82690afd8e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:28.729824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbdcb5f9ad940c197fabef60ed319d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.735009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1033/2031] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:28.738543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16555d29-8746-4821-8d21-82690afd8e4f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:28.741260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbdcb5f9ad940c197fabef60ed319d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.742026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbdcb5f9ad940c197fabef60ed319d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:28.747167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16555d29-8746-4821-8d21-82690afd8e4f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:28.748906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 999/2032] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:27 2022] DELETE /identity/v3/users/ffbdcb5f9ad940c197fabef60ed319d9 => generated 0 bytes in 1212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:28.749699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:28.753283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:28.770733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:28.813943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:28.818305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:28.818305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:28.819841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] PATH_INFO: `/v3/projects/522f920cb7c540c69746932fac6d63b6/users/079e7c3a6e02427d84574e2db1a572cb/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:28.824929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d93bab-30c7-45b6-afe8-e6441bd6341e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:28.829067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:28.829683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:28.829978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:28.830306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:28.832959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:28.842732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=522f920cb7c540c69746932fac6d63b6, user_id=079e7c3a6e02427d84574e2db1a572cb, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:28.845742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:28.846471 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:28.852886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:28.860012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1000/2033] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:28.947905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1034/2034] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:28 2022] PUT /identity/v3/projects/522f920cb7c540c69746932fac6d63b6/users/079e7c3a6e02427d84574e2db1a572cb/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:28.954050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:28.954050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:28.966579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd85a5-cbc8-42c2-a41b-4896b13fcc63 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.010880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:29.047479 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6485d5-92ee-4fd5-a8a1-4b09f86070ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.048147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6485d5-92ee-4fd5-a8a1-4b09f86070ab admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.048472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6485d5-92ee-4fd5-a8a1-4b09f86070ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.048748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6485d5-92ee-4fd5-a8a1-4b09f86070ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.049453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6485d5-92ee-4fd5-a8a1-4b09f86070ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.052068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6485d5-92ee-4fd5-a8a1-4b09f86070ab admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.065810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1001/2035] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:29.067861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.078962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cfc94-6c93-4a69-bc14-688992ba3aff admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.084869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.085911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.086550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.086903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] PATH_INFO: `/v3/projects/522f920cb7c540c69746932fac6d63b6/users/079e7c3a6e02427d84574e2db1a572cb/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.092808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=522f920cb7c540c69746932fac6d63b6, user_id=079e7c3a6e02427d84574e2db1a572cb, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.095670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.107931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1035/2036] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:29 2022] PUT /identity/v3/projects/522f920cb7c540c69746932fac6d63b6/users/079e7c3a6e02427d84574e2db1a572cb/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:29.119925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.120219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.120498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.154547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] MFA Rules not processed for user `079e7c3a6e02427d84574e2db1a572cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:29.156348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.230507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1002/2037] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:29.241294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.242999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.267561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.270188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.326704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e194a-e4e3-4789-b40e-80ccec324265 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.332353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.332353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.332353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.332792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.341394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce928-7572-4806-947b-9eff7ec9ec1d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.342399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.345322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.346397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.347048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.348063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.348506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] PATH_INFO: `/v3/projects/cc1e26e5821245a5997f5e7aa4ddd4f5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.349598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1e26e5821245a5997f5e7aa4ddd4f5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.351540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.357169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1036/2038] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:29.363626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.364983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.385669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e3bf-4eae-4d23-a206-ecfc310d1bf7 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.396053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.397020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.397020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.397273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] PATH_INFO: `/v3/projects/c22468b313144406a260b57953b0ecca` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.398593 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22468b313144406a260b57953b0ecca)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.401123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.520755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1e26e5821245a5997f5e7aa4ddd4f5', 'request_id': 'req-215776c1-cc49-47c7-b7b7-d276eb2c4466'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.527235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1e26e5821245a5997f5e7aa4ddd4f5', 'request_id': 'req-215776c1-cc49-47c7-b7b7-d276eb2c4466'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.530514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:29.535434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1e26e5821245a5997f5e7aa4ddd4f5', 'request_id': 'req-215776c1-cc49-47c7-b7b7-d276eb2c4466'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.547934 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22468b313144406a260b57953b0ecca', 'request_id': 'req-14436b74-5920-448d-84ba-e612cd8c9c4b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.554208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22468b313144406a260b57953b0ecca', 'request_id': 'req-14436b74-5920-448d-84ba-e612cd8c9c4b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.599245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] The token cache is being invalidate because project cc1e26e5821245a5997f5e7aa4ddd4f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:29.599797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.600175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-215776c1-cc49-47c7-b7b7-d276eb2c4466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.601722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1003/2039] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:29 2022] DELETE /identity/v3/projects/cc1e26e5821245a5997f5e7aa4ddd4f5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:29.607479 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385e9de-7aea-41a7-a81d-528fc1fb9597 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.607766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385e9de-7aea-41a7-a81d-528fc1fb9597 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.608168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385e9de-7aea-41a7-a81d-528fc1fb9597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.627811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a385e9de-7aea-41a7-a81d-528fc1fb9597 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:29.629243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a385e9de-7aea-41a7-a81d-528fc1fb9597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.672969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22468b313144406a260b57953b0ecca', 'request_id': 'req-14436b74-5920-448d-84ba-e612cd8c9c4b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.685704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] The token cache is being invalidate because project c22468b313144406a260b57953b0ecca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:29.685988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.686670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.688196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1037/2040] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:29 2022] DELETE /identity/v3/projects/c22468b313144406a260b57953b0ecca => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:29.692859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.694427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.695688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1004/2041] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:29.704056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a766b-f54a-46fd-90b8-0b5a7b34c826 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.704470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a766b-f54a-46fd-90b8-0b5a7b34c826 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.704752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a766b-f54a-46fd-90b8-0b5a7b34c826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.708599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14436b74-5920-448d-84ba-e612cd8c9c4b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.715074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.715748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.716029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.716303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] PATH_INFO: `/v3/users/dc848ef4fb58440da832b9c5deafd4a9` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.717494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc848ef4fb58440da832b9c5deafd4a9)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.719514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.727488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3d6a766b-f54a-46fd-90b8-0b5a7b34c826 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:29.732778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3d6a766b-f54a-46fd-90b8-0b5a7b34c826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.734456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:29.736946 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1005/2042] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:29.741232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a766b-f54a-46fd-90b8-0b5a7b34c826 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.750229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a766b-f54a-46fd-90b8-0b5a7b34c826 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.757240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.757240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.757240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.757505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.758213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:29.758523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.760282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.773739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1006/2043] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:40:29.774604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.776258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.835156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96027b4b-a927-4a1c-a7cc-411c963a10a0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.838819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.839647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.839766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.840127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.842071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:29.904245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc848ef4fb58440da832b9c5deafd4a9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.904456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.905846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:29.915575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1007/2044] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:29.916372 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:29.920328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc848ef4fb58440da832b9c5deafd4a9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:29.930253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef8f05-a8df-4a33-b4b3-f497f4f8eb94 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:29.939518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:29.940356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:29.942267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:29.942774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:29.944093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:29.946785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.032565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc848ef4fb58440da832b9c5deafd4a9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.033410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc848ef4fb58440da832b9c5deafd4a9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.034184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc848ef4fb58440da832b9c5deafd4a9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.049135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1038/2045] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:29 2022] DELETE /identity/v3/users/dc848ef4fb58440da832b9c5deafd4a9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:30.068163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1008/2046] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:29 2022] POST /identity/v3/projects => generated 399 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:30.069378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:30.073410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:30.153473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f903f9-f78c-406a-8d2e-5a3d8625a7c0 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:30.223450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:30.224076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.224364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.224624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.225291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:30.227264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.324648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1039/2047] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:30 2022] POST /identity/v3/users => generated 300 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:30.328816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:30.335008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:30.340557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5741e03-d83b-4519-a174-74cfdb7bac95 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:30.345059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:30.345681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.346021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.346335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.347618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:30.349332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.361407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1009/2048] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:30.366399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:30.376667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c979529-f262-4b41-86e7-29d07b3dcc54 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:30.381400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbcfd82-2953-4a4f-a9ab-3ed3bca8b4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:30.381999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfd82-2953-4a4f-a9ab-3ed3bca8b4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.382314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfd82-2953-4a4f-a9ab-3ed3bca8b4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.382594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbcfd82-2953-4a4f-a9ab-3ed3bca8b4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.383288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcfd82-2953-4a4f-a9ab-3ed3bca8b4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:30.384885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbcfd82-2953-4a4f-a9ab-3ed3bca8b4c2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.394926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1040/2049] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:30.399111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:30.410683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75dd85-60d1-4f9a-9d10-f76b3db8e064 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:30.417509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:30.418255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.418539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.418815 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] PATH_INFO: `/v3/projects/24881449468047cba55d92e69718998b/users/db7ab4686195472598f392f0ec929739/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.437033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24881449468047cba55d92e69718998b, user_id=db7ab4686195472598f392f0ec929739, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:30.440245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.506822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1010/2050] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:30 2022] PUT /identity/v3/projects/24881449468047cba55d92e69718998b/users/db7ab4686195472598f392f0ec929739/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:30.518446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.518917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.519465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.554447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] MFA Rules not processed for user `db7ab4686195472598f392f0ec929739`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:30.555664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:30.560790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:30.633728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1041/2051] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:30.640048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:30.695446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:30.696855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:30.707397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626019c-0af1-49de-83ec-b2335d1897c5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:30.715304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154076c-a613-492d-90ee-e86aeee9d712 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:30.716354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:30.716993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.717273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.717612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.718272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:30.720190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.720797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:30.723417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.723633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.723926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] PATH_INFO: `/v3/projects/e9ca2ad15f8f44709e621d245a765bde` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.725887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ca2ad15f8f44709e621d245a765bde)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:30.728156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.811832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1011/2052] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:30 2022] POST /identity/v3/projects => generated 399 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:30.815402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:30.874721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ca2ad15f8f44709e621d245a765bde', 'request_id': 'req-f7e88eaa-acee-497e-be10-55ee615f2e91'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.879927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c87b7-2120-41b1-9e15-2a23b344fa7c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:30.881088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ca2ad15f8f44709e621d245a765bde', 'request_id': 'req-f7e88eaa-acee-497e-be10-55ee615f2e91'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.935784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:30.936402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.936684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.936960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:30.937615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:30.939554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:30.948047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:30.974034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ca2ad15f8f44709e621d245a765bde', 'request_id': 'req-f7e88eaa-acee-497e-be10-55ee615f2e91'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.982395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] The token cache is being invalidate because project e9ca2ad15f8f44709e621d245a765bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:30.982684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.983335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7e88eaa-acee-497e-be10-55ee615f2e91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:30.984888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1042/2053] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:30 2022] DELETE /identity/v3/projects/e9ca2ad15f8f44709e621d245a765bde => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:30.990007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e2a32-e70d-4629-81d8-9e43a7cfe2d8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:30.990417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e2a32-e70d-4629-81d8-9e43a7cfe2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:30.990628 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e2a32-e70d-4629-81d8-9e43a7cfe2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.010358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-bd9e2a32-e70d-4629-81d8-9e43a7cfe2d8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:31.011558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bd9e2a32-e70d-4629-81d8-9e43a7cfe2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:31.019867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1043/2054] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:31.027408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff5bc-8216-4085-bf00-11fc212af541 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.027685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff5bc-8216-4085-bf00-11fc212af541 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.027960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff5bc-8216-4085-bf00-11fc212af541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.052954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-0f6ff5bc-8216-4085-bf00-11fc212af541 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:31.054206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0f6ff5bc-8216-4085-bf00-11fc212af541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:31.061682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1012/2055] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:30 2022] POST /identity/v3/users => generated 306 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:31.064647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1044/2056] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:31.065541 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.066655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff5bc-8216-4085-bf00-11fc212af541 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.068126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:31.076177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff5bc-8216-4085-bf00-11fc212af541 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.080102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967a458-2a0b-41e9-9948-cc1c2496d486 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.081328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.081929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.082240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.082514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.083315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:31.083616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.085136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.085360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.085750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.086032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.086337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.087027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.088602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.097712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1045/2057] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:40:31.100035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1013/2058] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:31.101991 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.103976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.111983 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5a10b-6bae-4dd3-af6a-c65c497e4eb5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.113810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f2ac4-5e92-4f0e-adcf-bd37bd3b2b17 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.116837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.117456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.117737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.118009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.118689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.119197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.119849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.120129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.120403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] PATH_INFO: `/v3/projects/d15d6af292d24e3aa622817ad03b37ff/users/360af207f4334322ba8cbc0b7c6419ab/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.120596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.135895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15d6af292d24e3aa622817ad03b37ff, user_id=360af207f4334322ba8cbc0b7c6419ab, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.138597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.229387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1046/2059] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:31 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:31.233434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.239924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1014/2060] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:31 2022] PUT /identity/v3/projects/d15d6af292d24e3aa622817ad03b37ff/users/360af207f4334322ba8cbc0b7c6419ab/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:31.244092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.279958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:31.300217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad50374-5370-46b8-b5f9-1da7635ac35b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.311384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cab50-f925-407d-b3de-c4ef2fa2d890 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.318723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.318723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.318723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.318723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.319110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.319628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.330171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1015/2061] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:31.334488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.344155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64add351-658d-4b16-aaf2-4982154ff5d9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.349779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c191af-0dc4-47cb-8830-ffdd324c6076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.350600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c191af-0dc4-47cb-8830-ffdd324c6076 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.350996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c191af-0dc4-47cb-8830-ffdd324c6076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.351279 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c191af-0dc4-47cb-8830-ffdd324c6076 admin admin] PATH_INFO: `/v3/projects/d15d6af292d24e3aa622817ad03b37ff/users/360af207f4334322ba8cbc0b7c6419ab/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.353190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c191af-0dc4-47cb-8830-ffdd324c6076 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15d6af292d24e3aa622817ad03b37ff, user_id=360af207f4334322ba8cbc0b7c6419ab, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.355973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c191af-0dc4-47cb-8830-ffdd324c6076 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.364407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.365179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.365448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.368022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.368725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.370229 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1016/2062] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:31 2022] PUT /identity/v3/projects/d15d6af292d24e3aa622817ad03b37ff/users/360af207f4334322ba8cbc0b7c6419ab/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:31.370736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.378091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.378461 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.378743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.407003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] MFA Rules not processed for user `360af207f4334322ba8cbc0b7c6419ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:31.408216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:31.476408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1047/2063] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:31 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:31.477315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.488599 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0fee9-1d01-43a8-8a28-2ec7f2ba6f51 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.490403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1017/2064] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:31.495216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.496699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:31.530313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:31.562775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.563516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.563798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.564070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.564769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.566425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.571402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58083443-dc70-4e27-a3bb-2a159e8ee60e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.575359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.575975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.582281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.582281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.582496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1048/2065] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:31.582496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.587854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.589288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.593589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e778f-6c64-4bd5-8ff7-21f7d3c82d80 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.600289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042d3a3-26aa-46ab-8f7d-d422941d06de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.600914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1018/2066] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:31.602404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042d3a3-26aa-46ab-8f7d-d422941d06de admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.602708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042d3a3-26aa-46ab-8f7d-d422941d06de admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.605237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042d3a3-26aa-46ab-8f7d-d422941d06de admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.605237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042d3a3-26aa-46ab-8f7d-d422941d06de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.605678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042d3a3-26aa-46ab-8f7d-d422941d06de admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.617339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1049/2067] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:31.621861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.636068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848d417-ef4b-450e-a4a8-5f0bcb5fcbb5 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.639221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.639911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.640190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.640732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] PATH_INFO: `/v3/projects/244a6ea63b2b4fde8ae3d4d378a5c5c3/users/cc550089c5e04a808bad501a99c4a248/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.655866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244a6ea63b2b4fde8ae3d4d378a5c5c3, user_id=cc550089c5e04a808bad501a99c4a248, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.658756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.758273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1019/2068] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:31 2022] PUT /identity/v3/projects/244a6ea63b2b4fde8ae3d4d378a5c5c3/users/cc550089c5e04a808bad501a99c4a248/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:31.771145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fdb0-1663-4c52-8881-785a62709024 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.771618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fdb0-1663-4c52-8881-785a62709024 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.771734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384fdb0-1663-4c52-8881-785a62709024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.799242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9384fdb0-1663-4c52-8881-785a62709024 None None] MFA Rules not processed for user `cc550089c5e04a808bad501a99c4a248`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:31.800450 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9384fdb0-1663-4c52-8881-785a62709024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:31.851390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:31.858693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.868813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1050/2069] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:31.873780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384fdb0-1663-4c52-8881-785a62709024 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:31.921484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa866767-7962-4791-9329-b40de78d7bc4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.925189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.925800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.926075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.926377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.928186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:31.937593 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384fdb0-1663-4c52-8881-785a62709024 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:31.943142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:31.944128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:31.944128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:31.944315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:31.944965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.946963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:31.994826 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:31.996183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:32.005903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1020/2070] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:32.072127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1051/2071] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:31 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:32.073356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:32.085405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756f83f-90fd-40e2-a31e-539990d4cc02 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:32.141230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:32.149048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:32.149674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:32.149953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:32.150262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:32.150937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:32.152857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:32.260152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1021/2072] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:32 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:32.261268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:32.272164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfccca4-fbe3-4836-be86-5e7474d0c6a9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:32.276710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:32.277429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:32.277632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:32.277932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:32.278664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:32.280250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:32.291025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1052/2073] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:32.294974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:32.306321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811edeb9-500e-4266-a46b-19d02e596699 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:32.311255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de744605-5990-4823-b004-634347b66f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:32.311882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744605-5990-4823-b004-634347b66f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:32.312168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744605-5990-4823-b004-634347b66f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:32.312444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de744605-5990-4823-b004-634347b66f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:32.313146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de744605-5990-4823-b004-634347b66f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:32.314781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de744605-5990-4823-b004-634347b66f82 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:32.325282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1022/2074] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:32.329787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:32.341129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb062dd7-3136-4fed-a2b6-b1fa6a0356f3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:32.345740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:32.346434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:32.346715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:32.347670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] PATH_INFO: `/v3/projects/a4a3f494c07d45368f9af039051ba6c5/users/ac24d30eb5eb442aa802901e5aa28b7f/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:32.364307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a3f494c07d45368f9af039051ba6c5, user_id=ac24d30eb5eb442aa802901e5aa28b7f, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:32.367074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:32.461506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1053/2075] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:32 2022] PUT /identity/v3/projects/a4a3f494c07d45368f9af039051ba6c5/users/ac24d30eb5eb442aa802901e5aa28b7f/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:32.474354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e133e0-0e46-4dc3-b608-8fb505f506d1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:32.474779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e133e0-0e46-4dc3-b608-8fb505f506d1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:32.475060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e133e0-0e46-4dc3-b608-8fb505f506d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:32.504309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-16e133e0-0e46-4dc3-b608-8fb505f506d1 None None] MFA Rules not processed for user `ac24d30eb5eb442aa802901e5aa28b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:32.505534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-16e133e0-0e46-4dc3-b608-8fb505f506d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:32.591137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1023/2076] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:32.595410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:32.662374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad61d9c-04c7-4a9a-ae47-329ecaa574c2 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:32.667080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:32.667697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:32.667959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:32.668241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:32.668913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:32.670873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:32.796190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1054/2077] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:32 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:32.801351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e133e0-0e46-4dc3-b608-8fb505f506d1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:32.812114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e133e0-0e46-4dc3-b608-8fb505f506d1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:32.872707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:32.873337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:32.873621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:32.873892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:32.874595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:32.876519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:32.988042 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1024/2078] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:32 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:32.992651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:33.002898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d576a-8c3c-48dd-ace8-9210d368091e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:33.008372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:33.008985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:33.009264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:33.009539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:33.010292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:33.011886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:33.023481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1055/2079] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:33.027754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:33.038273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb6c1e-8df6-4f64-88a7-e39649697afd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:33.047167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:33.047167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:33.047167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:33.047167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] PATH_INFO: `/v3/projects/f2581794169a462d8b86abf87881b28d/users/356ec131a4e04d9593d8f9b5d487369f/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:33.059746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2581794169a462d8b86abf87881b28d, user_id=356ec131a4e04d9593d8f9b5d487369f, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:33.062441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:33.153012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1025/2080] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:33 2022] PUT /identity/v3/projects/f2581794169a462d8b86abf87881b28d/users/356ec131a4e04d9593d8f9b5d487369f/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:33.160331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:33.172725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ccf278-9a06-4f93-97e4-92673001be81 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:33.192814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:33.244584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04177c71-2f1f-4bf9-aeed-4e8ad00d73d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:33.245177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04177c71-2f1f-4bf9-aeed-4e8ad00d73d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:33.245451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04177c71-2f1f-4bf9-aeed-4e8ad00d73d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:33.245714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04177c71-2f1f-4bf9-aeed-4e8ad00d73d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:33.246467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04177c71-2f1f-4bf9-aeed-4e8ad00d73d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:33.248116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04177c71-2f1f-4bf9-aeed-4e8ad00d73d0 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:33.258102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1056/2081] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:33.262258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:33.272539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d529-afe1-49cf-9815-1a8218d6d993 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:33.277888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:33.278650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:33.278934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:33.279209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] PATH_INFO: `/v3/projects/f2581794169a462d8b86abf87881b28d/users/356ec131a4e04d9593d8f9b5d487369f/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:33.281220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2581794169a462d8b86abf87881b28d, user_id=356ec131a4e04d9593d8f9b5d487369f, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:33.283911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:33.294688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1026/2082] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:33 2022] PUT /identity/v3/projects/f2581794169a462d8b86abf87881b28d/users/356ec131a4e04d9593d8f9b5d487369f/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:33.305794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3670209-6693-4155-b4cd-8b87e583f41b None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:33.306009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3670209-6693-4155-b4cd-8b87e583f41b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:33.306315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3670209-6693-4155-b4cd-8b87e583f41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:33.334640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a3670209-6693-4155-b4cd-8b87e583f41b None None] MFA Rules not processed for user `356ec131a4e04d9593d8f9b5d487369f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:33.335866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a3670209-6693-4155-b4cd-8b87e583f41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:33.402996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1057/2083] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:33.415071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:33.416735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:33.435193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:33.474682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546116f1-6ce1-4763-8030-deced4d2f6f3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:33.478803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:33.479446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:33.479723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:33.479994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:33.539060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:33.540545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:33.554888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1027/2084] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:40.701213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3670209-6693-4155-b4cd-8b87e583f41b None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:40.713520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3670209-6693-4155-b4cd-8b87e583f41b None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:40.717665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:40.718317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:40.718597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:40.718873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:40.779275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:40.780726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:40.789963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1058/2085] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:42.511336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:42.520713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a9c71-b9f5-47ac-bf96-229733022fef service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:42.528359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6997d-20c3-4227-ba0d-c47143f1cde7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:42.528988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6997d-20c3-4227-ba0d-c47143f1cde7 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:42.530285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6997d-20c3-4227-ba0d-c47143f1cde7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:42.530285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6997d-20c3-4227-ba0d-c47143f1cde7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:42.539273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6997d-20c3-4227-ba0d-c47143f1cde7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:42.540708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6997d-20c3-4227-ba0d-c47143f1cde7 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:42.552577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1028/2086] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:43.078842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:43.124927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:43.150108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f72cb-d81f-4dd8-ab20-3061a8a1b038 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:43.155253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:43.155956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:43.156237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:43.156517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] PATH_INFO: `/v3/users/db7ab4686195472598f392f0ec929739` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:43.158795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7ab4686195472598f392f0ec929739)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:43.167087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:43.366760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7ab4686195472598f392f0ec929739'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:43.378413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7ab4686195472598f392f0ec929739'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:43.437607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7ab4686195472598f392f0ec929739'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:43.438165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7ab4686195472598f392f0ec929739'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:43.438862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7ab4686195472598f392f0ec929739'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:43.452343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1059/2087] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:43 2022] DELETE /identity/v3/users/db7ab4686195472598f392f0ec929739 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:43.457122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:43.457407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:43.457679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:43.476808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:43.478061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:43.536493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1029/2088] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:43.546301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:43.547724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:43.609761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d633-11e8-4054-a9f9-3a0819731d61 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:43.613406 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:43.614038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:43.614830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:43.615111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:43.624370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:43.625804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:43.636089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1060/2089] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:44.067692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:44.069402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:44.094650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288b47-377c-4193-a6d3-56f74ee6e4e0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:44.096175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:44.096828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:44.097104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:44.097374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] PATH_INFO: `/v3/projects/24881449468047cba55d92e69718998b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:44.098811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24881449468047cba55d92e69718998b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:44.101556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:44.209492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24881449468047cba55d92e69718998b', 'request_id': 'req-96da8525-fb56-4063-a680-7fe7105364f0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.211373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:44.215263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24881449468047cba55d92e69718998b', 'request_id': 'req-96da8525-fb56-4063-a680-7fe7105364f0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.221398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24881449468047cba55d92e69718998b', 'request_id': 'req-96da8525-fb56-4063-a680-7fe7105364f0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.335183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] The token cache is being invalidate because project 24881449468047cba55d92e69718998b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:44.337626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.337626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.338182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1030/2090] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:44 2022] DELETE /identity/v3/projects/24881449468047cba55d92e69718998b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:44.342093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:44.352524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d636e2-ad04-45fb-8b38-f4e1ba0c2ed8 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:44.367127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:44.368735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:44.426560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:44.435716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:44.436365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:44.436646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:44.436913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] PATH_INFO: `/v3/users/360af207f4334322ba8cbc0b7c6419ab` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:44.438676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=360af207f4334322ba8cbc0b7c6419ab)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:44.441329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:44.483574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da8525-fb56-4063-a680-7fe7105364f0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:44.492686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:44.493351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:44.493635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:44.493909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] PATH_INFO: `/v3/users/712acc57d05b432fa9f5d7a7c09dec39` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:44.495144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=712acc57d05b432fa9f5d7a7c09dec39)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:44.497094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:44.664731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '360af207f4334322ba8cbc0b7c6419ab'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.676608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '360af207f4334322ba8cbc0b7c6419ab'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.743264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712acc57d05b432fa9f5d7a7c09dec39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.750886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '360af207f4334322ba8cbc0b7c6419ab'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.751509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '360af207f4334322ba8cbc0b7c6419ab'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.752274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-301b1753-8450-4281-aa23-0051a78b1fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '360af207f4334322ba8cbc0b7c6419ab'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.756104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712acc57d05b432fa9f5d7a7c09dec39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.769059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1061/2091] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:44 2022] DELETE /identity/v3/users/360af207f4334322ba8cbc0b7c6419ab => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:44.838027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712acc57d05b432fa9f5d7a7c09dec39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.845640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712acc57d05b432fa9f5d7a7c09dec39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.846368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712acc57d05b432fa9f5d7a7c09dec39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:44.852840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1031/2092] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:44 2022] DELETE /identity/v3/users/712acc57d05b432fa9f5d7a7c09dec39 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:44.857275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:44.857572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:44.857848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:44.877816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:44.882031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:44.938481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1062/2093] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:44.948719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:44.950133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:45.014787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a15129-2847-4b41-94d1-f7c2c8824e52 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.018613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.019342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.019685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.020031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.030602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.032066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.042887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1032/2094] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:45.216896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:45.218453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:45.237390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa18d50-ed3c-4f90-a0d8-b004eae24aee None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.244372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.245053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.245350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.245622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] PATH_INFO: `/v3/projects/d15d6af292d24e3aa622817ad03b37ff` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.247354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15d6af292d24e3aa622817ad03b37ff)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.251440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.391679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15d6af292d24e3aa622817ad03b37ff', 'request_id': 'req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.406078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15d6af292d24e3aa622817ad03b37ff', 'request_id': 'req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.472990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15d6af292d24e3aa622817ad03b37ff', 'request_id': 'req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.482656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:45.486506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] The token cache is being invalidate because project d15d6af292d24e3aa622817ad03b37ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:45.486668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:45.486799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.487486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e86a90c8-4a73-4c9d-951b-b8547eb4c3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.488197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:45.489367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1063/2095] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:45 2022] DELETE /identity/v3/projects/d15d6af292d24e3aa622817ad03b37ff => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:45.493714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405156c-72bd-4fe6-a4f8-f766ca5283c0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.494002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405156c-72bd-4fe6-a4f8-f766ca5283c0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.494309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405156c-72bd-4fe6-a4f8-f766ca5283c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.497998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:45.504228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc7114-213a-4959-b052-770010120d54 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.518773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e405156c-72bd-4fe6-a4f8-f766ca5283c0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:45.519351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e405156c-72bd-4fe6-a4f8-f766ca5283c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:45.582020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.582954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.583570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.583985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] PATH_INFO: `/v3/projects/41384c34da9f46e59a12bc80d794ac03` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.585247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] RBAC: Authorizing `identity:delete_project(project_id=41384c34da9f46e59a12bc80d794ac03)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.587510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.591882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1064/2096] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:45.599711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6cf4b-044d-4cd3-aed3-29de27fd8bc2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.600000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6cf4b-044d-4cd3-aed3-29de27fd8bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.601928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6cf4b-044d-4cd3-aed3-29de27fd8bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.623207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e1e6cf4b-044d-4cd3-aed3-29de27fd8bc2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:45.624518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e1e6cf4b-044d-4cd3-aed3-29de27fd8bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:45.633335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1065/2097] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:45.637498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6cf4b-044d-4cd3-aed3-29de27fd8bc2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:45.646895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6cf4b-044d-4cd3-aed3-29de27fd8bc2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.651589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.652192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.652471 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.652736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.653444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:45.653743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.655370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.667972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1066/2098] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:40:45.679009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:45.688823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7359d5-edc1-44fa-b81e-995fcdbaa47d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.694175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.694803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.695081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.695350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.696004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.697973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.711879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41384c34da9f46e59a12bc80d794ac03', 'request_id': 'req-378b3280-68fb-4910-bc4a-00052be796db'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.718659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41384c34da9f46e59a12bc80d794ac03', 'request_id': 'req-378b3280-68fb-4910-bc4a-00052be796db'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.725028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41384c34da9f46e59a12bc80d794ac03', 'request_id': 'req-378b3280-68fb-4910-bc4a-00052be796db'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.790737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] The token cache is being invalidate because project 41384c34da9f46e59a12bc80d794ac03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:45.791391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.791572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:45.793087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1033/2099] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:45 2022] DELETE /identity/v3/projects/41384c34da9f46e59a12bc80d794ac03 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:45.793322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1067/2100] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:45 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:45.796992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:45.797140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:45.798596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:45.798853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:45.861736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494da8bc-96b8-479e-9b25-b1acd4f6f622 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.862669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3280-68fb-4910-bc4a-00052be796db admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.867134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.867745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.868025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.868408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.868636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.869097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.869272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.869548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.869821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] PATH_INFO: `/v3/users/079e7c3a6e02427d84574e2db1a572cb` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.871003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=079e7c3a6e02427d84574e2db1a572cb)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.871195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.872904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.959068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1068/2101] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:45 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:45.963472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:45.974359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e679ff-b994-42ef-83ad-81529271729a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:45.979154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:45.979762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:45.980084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:45.980468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:45.981169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:45.982806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:45.995052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1069/2102] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:45.998960 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:46.008737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76d905-7dc3-4b5f-8746-88dca8e69498 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:46.013641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:46.014475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:46.014756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:46.015029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:46.015915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:46.017649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:46.029287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1070/2103] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:46.033728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:46.043823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a4f94-e81a-410d-b970-e8770aaca06e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:46.048738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229749d4-f968-42e4-a484-1bbb473b61f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:46.049447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229749d4-f968-42e4-a484-1bbb473b61f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:46.049731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229749d4-f968-42e4-a484-1bbb473b61f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:46.050006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229749d4-f968-42e4-a484-1bbb473b61f0 admin admin] PATH_INFO: `/v3/projects/13d415416e294240b718f4886f462da4/users/972ea719298f458a90d289eef6c24bb5/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:46.066787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229749d4-f968-42e4-a484-1bbb473b61f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d415416e294240b718f4886f462da4, user_id=972ea719298f458a90d289eef6c24bb5, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:46.069564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229749d4-f968-42e4-a484-1bbb473b61f0 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:46.097144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079e7c3a6e02427d84574e2db1a572cb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:46.110251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079e7c3a6e02427d84574e2db1a572cb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:46.140533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:46.175395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1071/2104] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:46 2022] PUT /identity/v3/projects/13d415416e294240b718f4886f462da4/users/972ea719298f458a90d289eef6c24bb5/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:46.186401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:46.186708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:46.186990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:46.193981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:46.198510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079e7c3a6e02427d84574e2db1a572cb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:46.206129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079e7c3a6e02427d84574e2db1a572cb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:46.206840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079e7c3a6e02427d84574e2db1a572cb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:46.213048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1034/2105] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:45 2022] DELETE /identity/v3/users/079e7c3a6e02427d84574e2db1a572cb => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:46.216770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] MFA Rules not processed for user `972ea719298f458a90d289eef6c24bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:46.217691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:46.291193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1072/2106] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:46.303455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:46.305005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:46.364508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9974c90f-5ecd-4cb1-9ec0-e778d5c3b2e6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:46.368465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415aefb-bff2-44c1-90a5-c0bd1441b805 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:46.369079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415aefb-bff2-44c1-90a5-c0bd1441b805 service glance] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:46.369356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415aefb-bff2-44c1-90a5-c0bd1441b805 service glance] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:46.369628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415aefb-bff2-44c1-90a5-c0bd1441b805 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:46.378096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415aefb-bff2-44c1-90a5-c0bd1441b805 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:46.379664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415aefb-bff2-44c1-90a5-c0bd1441b805 service glance] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:46.388550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1035/2107] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:46.752954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:46.755258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:46.826830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a77a2-30ae-4f33-8e81-32793b367108 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:46.834894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:46.839725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:46.839725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:46.839725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] PATH_INFO: `/v3/projects/522f920cb7c540c69746932fac6d63b6` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:46.839725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=522f920cb7c540c69746932fac6d63b6)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:46.841957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:46.968031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '522f920cb7c540c69746932fac6d63b6', 'request_id': 'req-6517bb05-50e6-4ebf-8278-3aa21a069538'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:46.973754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '522f920cb7c540c69746932fac6d63b6', 'request_id': 'req-6517bb05-50e6-4ebf-8278-3aa21a069538'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:47.089530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '522f920cb7c540c69746932fac6d63b6', 'request_id': 'req-6517bb05-50e6-4ebf-8278-3aa21a069538'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:47.109678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] The token cache is being invalidate because project 522f920cb7c540c69746932fac6d63b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:47.110053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:47.111510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-6517bb05-50e6-4ebf-8278-3aa21a069538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:47.114213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1073/2108] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:46 2022] DELETE /identity/v3/projects/522f920cb7c540c69746932fac6d63b6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:47.119961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e27cc8-1a2b-4a83-889b-e1d4ac9533c1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.120430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e27cc8-1a2b-4a83-889b-e1d4ac9533c1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.120519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e27cc8-1a2b-4a83-889b-e1d4ac9533c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.145025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e3e27cc8-1a2b-4a83-889b-e1d4ac9533c1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:47.146610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e3e27cc8-1a2b-4a83-889b-e1d4ac9533c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:47.222872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1036/2109] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:47.230660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90397f59-136d-443b-94fb-a3ecef66140d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.230955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90397f59-136d-443b-94fb-a3ecef66140d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.231233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90397f59-136d-443b-94fb-a3ecef66140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.252752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-90397f59-136d-443b-94fb-a3ecef66140d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:47.254055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-90397f59-136d-443b-94fb-a3ecef66140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:47.263869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1074/2110] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:47.268173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e27cc8-1a2b-4a83-889b-e1d4ac9533c1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:47.277767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e27cc8-1a2b-4a83-889b-e1d4ac9533c1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:47.282583 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:47.283197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.283476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.283748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.284443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:47.284747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:47.286381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:47.300028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1037/2111] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:47.307486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90397f59-136d-443b-94fb-a3ecef66140d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:47.316829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90397f59-136d-443b-94fb-a3ecef66140d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:47.322399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:47.322914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.323201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.323482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.324147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:47.326074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:47.362597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:47.460520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1075/2112] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:47 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:47.465050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:47.489391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378109a-22a4-4419-a77d-2b0c89850183 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:47.559909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:47.581719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:47.582469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.582874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.583218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.584319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:47.586543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:47.675529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1038/2113] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:47 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:47.676801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:47.687995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf7c7-e4b6-4b67-ba63-3e313c08a750 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:47.692609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:47.694497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.694497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.694497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.694999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:47.696614 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:47.707657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1076/2114] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:47.711633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:47.721766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2bb73-2073-412f-93af-1bbcc15549a5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:47.727242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b5b72-fbf8-42cd-a55d-9bf93ccf085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:47.727828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b5b72-fbf8-42cd-a55d-9bf93ccf085d admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.728108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b5b72-fbf8-42cd-a55d-9bf93ccf085d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.728384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b5b72-fbf8-42cd-a55d-9bf93ccf085d admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.729083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b5b72-fbf8-42cd-a55d-9bf93ccf085d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:47.730837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b5b72-fbf8-42cd-a55d-9bf93ccf085d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:47.742987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1039/2115] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:47.746729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:47.757707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c9208-48f1-418f-b30c-fbb573e4db3a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:47.762377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:47.763043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.763326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.763605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] PATH_INFO: `/v3/projects/ffd76d245a444190b1a7125944b0cfa0/users/fde6cc4e33bc4669b5f9939e66a5741c/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.781025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd76d245a444190b1a7125944b0cfa0, user_id=fde6cc4e33bc4669b5f9939e66a5741c, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:47.783833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:47.838796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1077/2116] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:47 2022] PUT /identity/v3/projects/ffd76d245a444190b1a7125944b0cfa0/users/fde6cc4e33bc4669b5f9939e66a5741c/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:47.849169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798175b-4eb2-44b3-9b68-99459ad5fcd4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:47.849400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798175b-4eb2-44b3-9b68-99459ad5fcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:47.849736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798175b-4eb2-44b3-9b68-99459ad5fcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:47.878089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5798175b-4eb2-44b3-9b68-99459ad5fcd4 None None] MFA Rules not processed for user `fde6cc4e33bc4669b5f9939e66a5741c`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:47.879394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5798175b-4eb2-44b3-9b68-99459ad5fcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:47.948794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1040/2117] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:47.953622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.025623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cab9e-2fb1-4c3b-b27c-7059184cc6de admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.044699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.060260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.060260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.060260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.060260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.060260 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.168953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1078/2118] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:47 2022] POST /identity/v3/projects => generated 381 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:48.172838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798175b-4eb2-44b3-9b68-99459ad5fcd4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.184897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798175b-4eb2-44b3-9b68-99459ad5fcd4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.264377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.265374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.265374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.265581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.266237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.268213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.357500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1041/2119] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:48 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:48.358739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.369429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820efaa-2603-4864-84e9-aefb5947c351 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.376817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.377458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.377746 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.378018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.378762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.380408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.397768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1079/2120] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:40:48.402419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.416002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024bb5d-810e-4565-bcaa-2bd68cae4098 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.421770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.422702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.423003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.423291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] PATH_INFO: `/v3/projects/660fbb949cf845dc8517d19f25c06b63/users/3ef86bc1df154257b3628a05cb2db350/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.442054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=660fbb949cf845dc8517d19f25c06b63, user_id=3ef86bc1df154257b3628a05cb2db350, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.445997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.518924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1042/2121] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:48 2022] PUT /identity/v3/projects/660fbb949cf845dc8517d19f25c06b63/users/3ef86bc1df154257b3628a05cb2db350/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:48.525313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.538464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e5b92-6882-40df-9688-cf221b5cb43b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.579262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.582246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:48.606113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:48.621257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.627560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.627560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.627560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.627560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.627560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.647245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1080/2122] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:48.649393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.667042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1bf2-46e6-4ee7-bc62-41d7dc49bbde admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.674628 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84133fc5-07cc-42cb-94e9-525f53a233ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.675318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84133fc5-07cc-42cb-94e9-525f53a233ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.675601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84133fc5-07cc-42cb-94e9-525f53a233ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.675889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84133fc5-07cc-42cb-94e9-525f53a233ea admin admin] PATH_INFO: `/v3/projects/660fbb949cf845dc8517d19f25c06b63/users/3ef86bc1df154257b3628a05cb2db350/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.676318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0733ba-2ad4-4abf-93fb-58cd4d816d3c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.678453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84133fc5-07cc-42cb-94e9-525f53a233ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=660fbb949cf845dc8517d19f25c06b63, user_id=3ef86bc1df154257b3628a05cb2db350, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.681119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84133fc5-07cc-42cb-94e9-525f53a233ea admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.683849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.684832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.685419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.685715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] PATH_INFO: `/v3/users/cc550089c5e04a808bad501a99c4a248` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.687091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc550089c5e04a808bad501a99c4a248)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.690248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.692516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1081/2123] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:48 2022] PUT /identity/v3/projects/660fbb949cf845dc8517d19f25c06b63/users/3ef86bc1df154257b3628a05cb2db350/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:48.704528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.704838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.705113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.739524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] MFA Rules not processed for user `3ef86bc1df154257b3628a05cb2db350`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:48.743359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:48.750089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:48.832080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1082/2124] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:48.843157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:48.844750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:48.874286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc550089c5e04a808bad501a99c4a248'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:48.905346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc550089c5e04a808bad501a99c4a248'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:48.920276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf9d6-54ef-4c2c-a19d-123bdeb89243 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:48.924177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f7ff9-6340-4007-bac7-b3dfb77055c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:48.924796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7ff9-6340-4007-bac7-b3dfb77055c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:48.925073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7ff9-6340-4007-bac7-b3dfb77055c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:48.925342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f7ff9-6340-4007-bac7-b3dfb77055c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:48.939570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7ff9-6340-4007-bac7-b3dfb77055c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:48.941724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f7ff9-6340-4007-bac7-b3dfb77055c0 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:48.954475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1083/2125] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:48.990681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc550089c5e04a808bad501a99c4a248'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:48.998639 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc550089c5e04a808bad501a99c4a248'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:48.999315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc550089c5e04a808bad501a99c4a248'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:49.005551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1043/2126] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:48 2022] DELETE /identity/v3/users/cc550089c5e04a808bad501a99c4a248 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:49.010507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:49.010820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:49.011095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:49.033180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:49.033180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:49.104428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1084/2127] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:49.114575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:49.116185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:49.203433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87a1c8-e9c4-4c83-b2ae-e5cb5edf0004 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:49.208984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:49.210356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:49.210636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:49.210907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:49.236564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:49.237302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:49.249932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1044/2128] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:40:49.686434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:49.688120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:49.702732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862bc3-46fe-4ae6-b791-04fd13711b42 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:49.714016 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:49.714797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:49.715146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:49.715489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] PATH_INFO: `/v3/users/fde6cc4e33bc4669b5f9939e66a5741c` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:49.716715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde6cc4e33bc4669b5f9939e66a5741c)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:49.726085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:49.869456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:49.872143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:49.895008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4b522-0bbe-4ac6-9c28-979707dd6131 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:49.900109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:49.900542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:49.900902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:49.901075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] PATH_INFO: `/v3/projects/244a6ea63b2b4fde8ae3d4d378a5c5c3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:49.902374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244a6ea63b2b4fde8ae3d4d378a5c5c3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:49.904180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:49.925612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:49.933110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde6cc4e33bc4669b5f9939e66a5741c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:49.953004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde6cc4e33bc4669b5f9939e66a5741c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.019693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde6cc4e33bc4669b5f9939e66a5741c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.020387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde6cc4e33bc4669b5f9939e66a5741c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.022966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-964d8eb6-ea2e-4865-93d5-e0a58ed18090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde6cc4e33bc4669b5f9939e66a5741c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.029134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:50.045307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1085/2129] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:49 2022] DELETE /identity/v3/users/fde6cc4e33bc4669b5f9939e66a5741c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:50.049949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:50.050464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:50.050926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:50.054648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244a6ea63b2b4fde8ae3d4d378a5c5c3', 'request_id': 'req-261af380-e2a1-4e92-983b-e76d2013c297'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.061179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244a6ea63b2b4fde8ae3d4d378a5c5c3', 'request_id': 'req-261af380-e2a1-4e92-983b-e76d2013c297'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.068620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244a6ea63b2b4fde8ae3d4d378a5c5c3', 'request_id': 'req-261af380-e2a1-4e92-983b-e76d2013c297'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.071807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:50.073045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:50.147379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] The token cache is being invalidate because project 244a6ea63b2b4fde8ae3d4d378a5c5c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:50.147762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1086/2130] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:50.149255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.149996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.153875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1045/2131] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:49 2022] DELETE /identity/v3/projects/244a6ea63b2b4fde8ae3d4d378a5c5c3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:50.157828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:50.160586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:50.179893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba85dc-df13-4d61-b262-85ad3313a11b None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:50.181767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:50.183346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:50.191200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:50.191868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:50.192165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:50.192427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] PATH_INFO: `/v3/users/ac24d30eb5eb442aa802901e5aa28b7f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:50.197607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac24d30eb5eb442aa802901e5aa28b7f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:50.199706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:50.281994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261af380-e2a1-4e92-983b-e76d2013c297 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:50.349553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:50.350524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:50.351010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:50.351437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:50.354869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:50.376916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:50.378440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:50.389426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1046/2132] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:50.437539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac24d30eb5eb442aa802901e5aa28b7f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.478392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac24d30eb5eb442aa802901e5aa28b7f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.557577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac24d30eb5eb442aa802901e5aa28b7f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.558436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac24d30eb5eb442aa802901e5aa28b7f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.559229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac24d30eb5eb442aa802901e5aa28b7f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:50.599964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1087/2133] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:50 2022] DELETE /identity/v3/users/ac24d30eb5eb442aa802901e5aa28b7f => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:50.904343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:50.906445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:50.927470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f9ed5-6490-4c68-9fcc-6984ee0be2d9 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:51.002674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:51.003325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:51.003611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:51.003888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] PATH_INFO: `/v3/projects/ffd76d245a444190b1a7125944b0cfa0` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:51.005070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd76d245a444190b1a7125944b0cfa0)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:51.007254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:51.032466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:51.141772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd76d245a444190b1a7125944b0cfa0', 'request_id': 'req-96718275-7dcc-4540-a684-dd4177260508'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.147749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd76d245a444190b1a7125944b0cfa0', 'request_id': 'req-96718275-7dcc-4540-a684-dd4177260508'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.154177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd76d245a444190b1a7125944b0cfa0', 'request_id': 'req-96718275-7dcc-4540-a684-dd4177260508'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.274336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:51.276256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:51.290555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05582739-01ad-4527-b266-cf5b65cff264 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:51.328168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:51.341553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] The token cache is being invalidate because project ffd76d245a444190b1a7125944b0cfa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:51.342001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.342495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-96718275-7dcc-4540-a684-dd4177260508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.344064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1047/2134] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:50 2022] DELETE /identity/v3/projects/ffd76d245a444190b1a7125944b0cfa0 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:51.345490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda4fb9-59c2-47da-af70-2472dd3743f4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:51.345959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda4fb9-59c2-47da-af70-2472dd3743f4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:51.346266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda4fb9-59c2-47da-af70-2472dd3743f4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:51.347597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1048/2135] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:40:51.348290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda4fb9-59c2-47da-af70-2472dd3743f4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:51.350054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0eda4fb9-59c2-47da-af70-2472dd3743f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:51.361350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:51.365816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:51.366110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:51.366412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] PATH_INFO: `/v3/projects/a4a3f494c07d45368f9af039051ba6c5` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:51.368549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a3f494c07d45368f9af039051ba6c5)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:51.370774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:51.423008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda4fb9-59c2-47da-af70-2472dd3743f4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:51.428844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:51.429445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:51.429869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:51.434425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] PATH_INFO: `/v3/users/3ef86bc1df154257b3628a05cb2db350` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:51.435832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef86bc1df154257b3628a05cb2db350)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:51.438630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:51.489305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a3f494c07d45368f9af039051ba6c5', 'request_id': 'req-698bc3b0-c56d-448e-a396-593c7497865e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.495242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a3f494c07d45368f9af039051ba6c5', 'request_id': 'req-698bc3b0-c56d-448e-a396-593c7497865e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.569792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a3f494c07d45368f9af039051ba6c5', 'request_id': 'req-698bc3b0-c56d-448e-a396-593c7497865e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.581721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] The token cache is being invalidate because project a4a3f494c07d45368f9af039051ba6c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:51.582006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.582678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-698bc3b0-c56d-448e-a396-593c7497865e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.584170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1088/2136] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:51 2022] DELETE /identity/v3/projects/a4a3f494c07d45368f9af039051ba6c5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:51.585647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:51.586307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:51.587112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:51.600428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef86bc1df154257b3628a05cb2db350'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.605857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:51.607101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:51.612456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef86bc1df154257b3628a05cb2db350'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.670756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1089/2137] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:40:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:51.671604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:51.672862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:51.723015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef86bc1df154257b3628a05cb2db350'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.730496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef86bc1df154257b3628a05cb2db350'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.731249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef86bc1df154257b3628a05cb2db350'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:51.738207 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1049/2138] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:51 2022] DELETE /identity/v3/users/3ef86bc1df154257b3628a05cb2db350 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:51.739207 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:51.741402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:51.745694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a588-b0bb-4e6c-b591-0e18ede33571 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:51.809159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:51.809859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:51.810307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:51.810595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] PATH_INFO: `/v3/users/37554602ef6d461b97660ad6b337ed98` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:51.811915 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] RBAC: Authorizing `identity:delete_user(user_id=37554602ef6d461b97660ad6b337ed98)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:51.813874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:51.817138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a59ea-400c-4c4f-9663-df71c7632ec9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:51.823615 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:51.824266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:51.826068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:51.826068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] PATH_INFO: `/v3/users/356ec131a4e04d9593d8f9b5d487369f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:51.826068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] RBAC: Authorizing `identity:delete_user(user_id=356ec131a4e04d9593d8f9b5d487369f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:51.829470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:51.992011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:52.068061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:52.110911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37554602ef6d461b97660ad6b337ed98'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.115746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '356ec131a4e04d9593d8f9b5d487369f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.128413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37554602ef6d461b97660ad6b337ed98'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.151597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '356ec131a4e04d9593d8f9b5d487369f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.201846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37554602ef6d461b97660ad6b337ed98'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.202487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37554602ef6d461b97660ad6b337ed98'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.203093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37554602ef6d461b97660ad6b337ed98'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.216364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1090/2139] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:51 2022] DELETE /identity/v3/users/37554602ef6d461b97660ad6b337ed98 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:52.217145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:52.218489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.290331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d49dd9-620e-49c4-81ac-a65f5e3bc04d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:52.295356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:52.295978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.296259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.296530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.298630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.311387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:52.312982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:52.324242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1091/2140] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:52.324396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:52.328792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '356ec131a4e04d9593d8f9b5d487369f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.329244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.340550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '356ec131a4e04d9593d8f9b5d487369f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.340550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '356ec131a4e04d9593d8f9b5d487369f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.356478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1050/2141] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:51 2022] DELETE /identity/v3/users/356ec131a4e04d9593d8f9b5d487369f => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:52.410030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d676870-5dc9-4d15-aa1a-6f8e4fd511db service placement] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:52.479202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:52.479853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.480138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.480415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] PATH_INFO: `/v3/projects/660fbb949cf845dc8517d19f25c06b63` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.481592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=660fbb949cf845dc8517d19f25c06b63)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:52.483598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:52.560097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:52.568958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.581890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720faf7-971a-454d-8268-f63c25f4069f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:52.588291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:52.589644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.590077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.590530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] PATH_INFO: `/v3/projects/bbedb0ad0538471e98b0641598f08c17` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.594294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbedb0ad0538471e98b0641598f08c17)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:52.596749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:52.610274 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '660fbb949cf845dc8517d19f25c06b63', 'request_id': 'req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.619071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '660fbb949cf845dc8517d19f25c06b63', 'request_id': 'req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.716270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '660fbb949cf845dc8517d19f25c06b63', 'request_id': 'req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.730863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] The token cache is being invalidate because project 660fbb949cf845dc8517d19f25c06b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:52.731164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.743164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-76bb3c00-2d6d-4d9a-adac-8b5b952c8c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.743164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1092/2142] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:52 2022] DELETE /identity/v3/projects/660fbb949cf845dc8517d19f25c06b63 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:52.750116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366188e-2c61-424a-977b-fd71551ab8b0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.750553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366188e-2c61-424a-977b-fd71551ab8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.751153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366188e-2c61-424a-977b-fd71551ab8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.764724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbedb0ad0538471e98b0641598f08c17', 'request_id': 'req-70583f29-8f23-433f-953b-f10bc7df18e2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.770216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbedb0ad0538471e98b0641598f08c17', 'request_id': 'req-70583f29-8f23-433f-953b-f10bc7df18e2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.771402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d366188e-2c61-424a-977b-fd71551ab8b0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:52.772857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d366188e-2c61-424a-977b-fd71551ab8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.776837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbedb0ad0538471e98b0641598f08c17', 'request_id': 'req-70583f29-8f23-433f-953b-f10bc7df18e2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.833509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1093/2143] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:52.841559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcd4a9-be64-446b-a02b-13f760d675a9 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.841904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcd4a9-be64-446b-a02b-13f760d675a9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.842213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcd4a9-be64-446b-a02b-13f760d675a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.863631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-adbcd4a9-be64-446b-a02b-13f760d675a9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:52.864947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-adbcd4a9-be64-446b-a02b-13f760d675a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.865719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] The token cache is being invalidate because project bbedb0ad0538471e98b0641598f08c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:52.866172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.867445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:52.869797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1051/2144] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:52 2022] DELETE /identity/v3/projects/bbedb0ad0538471e98b0641598f08c17 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:40:52.870852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:52.872389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.878874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1094/2145] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:52.881990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd6d4f-4781-44b8-9f47-55ab382130d6 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.881990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd6d4f-4781-44b8-9f47-55ab382130d6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.881990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd6d4f-4781-44b8-9f47-55ab382130d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.889087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70583f29-8f23-433f-953b-f10bc7df18e2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:52.899264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:52.899849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.900151 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.900388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] PATH_INFO: `/v3/projects/f2581794169a462d8b86abf87881b28d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.901686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2581794169a462d8b86abf87881b28d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:52.904685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-80cd6d4f-4781-44b8-9f47-55ab382130d6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:52.906360 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-80cd6d4f-4781-44b8-9f47-55ab382130d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.906458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:52.918788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1095/2146] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:52.919622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd6d4f-4781-44b8-9f47-55ab382130d6 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:52.938116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd6d4f-4781-44b8-9f47-55ab382130d6 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:52.945193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7ad65-fbf2-447e-8242-41c4cef6c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:52.945844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7ad65-fbf2-447e-8242-41c4cef6c721 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.946119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7ad65-fbf2-447e-8242-41c4cef6c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.946421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7ad65-fbf2-447e-8242-41c4cef6c721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.947122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7ad65-fbf2-447e-8242-41c4cef6c721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:52.947421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7ad65-fbf2-447e-8242-41c4cef6c721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:52.949098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7ad65-fbf2-447e-8242-41c4cef6c721 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:52.957938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:52.962962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1096/2147] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:52.969180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10c3c-f5ca-40cf-a907-d9e45e2c4908 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:52.969180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10c3c-f5ca-40cf-a907-d9e45e2c4908 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:52.969180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e10c3c-f5ca-40cf-a907-d9e45e2c4908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:52.987165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a1e10c3c-f5ca-40cf-a907-d9e45e2c4908 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:52.988373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a1e10c3c-f5ca-40cf-a907-d9e45e2c4908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:52.996954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1097/2148] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:52.997694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10c3c-f5ca-40cf-a907-d9e45e2c4908 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.008192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e10c3c-f5ca-40cf-a907-d9e45e2c4908 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.011565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.012144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.012824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.012824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.013303 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.015237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.033214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:53.052909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2581794169a462d8b86abf87881b28d', 'request_id': 'req-741242ee-a369-40ab-b11e-77c951f564be'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:53.058410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2581794169a462d8b86abf87881b28d', 'request_id': 'req-741242ee-a369-40ab-b11e-77c951f564be'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:53.064724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2581794169a462d8b86abf87881b28d', 'request_id': 'req-741242ee-a369-40ab-b11e-77c951f564be'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:53.140091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1098/2149] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:52 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:53.140091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.163959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] The token cache is being invalidate because project f2581794169a462d8b86abf87881b28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:53.164388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:53.165089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:53.166680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1052/2150] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:52 2022] DELETE /identity/v3/projects/f2581794169a462d8b86abf87881b28d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:53.167635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.169140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:53.202909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93004ec-1d5b-4623-9717-1e431cbc4d30 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.207814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331e2f8-aecf-40c3-8591-5cac77f7b096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.208405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e2f8-aecf-40c3-8591-5cac77f7b096 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.209178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e2f8-aecf-40c3-8591-5cac77f7b096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.209445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e2f8-aecf-40c3-8591-5cac77f7b096 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.210160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331e2f8-aecf-40c3-8591-5cac77f7b096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:53.210462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331e2f8-aecf-40c3-8591-5cac77f7b096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.212047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331e2f8-aecf-40c3-8591-5cac77f7b096 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.223527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1099/2151] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:53.224857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.225145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.225413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.236605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741242ee-a369-40ab-b11e-77c951f564be admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.241382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edde45e-0ef1-4136-863c-4e6f6f897d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.242007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edde45e-0ef1-4136-863c-4e6f6f897d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.242320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edde45e-0ef1-4136-863c-4e6f6f897d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.242597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edde45e-0ef1-4136-863c-4e6f6f897d5a admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.243255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edde45e-0ef1-4136-863c-4e6f6f897d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.244900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:53.245170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edde45e-0ef1-4136-863c-4e6f6f897d5a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.246124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:53.254236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1100/2152] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:53.255220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.256391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:53.268515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342eb7a-5bc2-4a6c-b270-5374916a72a3 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.274568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.275181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.275456 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.275725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.276388 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.278313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.327831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1053/2153] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:53.329091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca259f-3faa-4fe1-abe6-1e69ea36e5be None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.329377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca259f-3faa-4fe1-abe6-1e69ea36e5be None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.329656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca259f-3faa-4fe1-abe6-1e69ea36e5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.349131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-52ca259f-3faa-4fe1-abe6-1e69ea36e5be None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:53.351923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-52ca259f-3faa-4fe1-abe6-1e69ea36e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:53.359929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1054/2154] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:53.360758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca259f-3faa-4fe1-abe6-1e69ea36e5be None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.370053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca259f-3faa-4fe1-abe6-1e69ea36e5be None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.374883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.375497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.375782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.376052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.377236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.378483 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.387642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1101/2155] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:53.387642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.394130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1055/2156] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:53.394914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.405394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72abf4c-69b7-40da-8589-ea458fbfcadb admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.450690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62fe65-5c5c-4172-9474-6388af218f5e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.456167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.456802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.457390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.457667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.458403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:53.458752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.460971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.476382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1102/2157] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:53.476539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.476539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.476807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.476903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.476903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.476903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.476903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.483191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350bf85-fdb6-4f8c-9c9e-c2710bf5603b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.488031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.488697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.488899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.489171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.489957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.491583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.502768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1103/2158] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:53.503654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.514468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed875038-a60d-46f1-8f76-70501c2c007a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.519182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.519793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.520066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.520332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.521030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.523551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.563342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1056/2159] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:53.564376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.575708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b232d-7dc0-48d7-8a4c-82976c360dc4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.580849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.581502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.581781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.582060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] PATH_INFO: `/v3/projects/5ce49bc8030e4508a0f3b4bdf6d49ecd/users/1c161ed8c0814110b212bbfd32d62a85/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.598466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce49bc8030e4508a0f3b4bdf6d49ecd, user_id=1c161ed8c0814110b212bbfd32d62a85, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.601514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.639225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1104/2160] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:53.640252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.678254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1057/2161] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:53 2022] PUT /identity/v3/projects/5ce49bc8030e4508a0f3b4bdf6d49ecd/users/1c161ed8c0814110b212bbfd32d62a85/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:53.679075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.708365 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97a6ac-65be-4b08-bc78-7c26392156f9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.754519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d18a-7d04-4125-ba5e-c252727939fd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.759192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:53.763426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.764198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.764576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.764947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.765773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.768013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.771820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97422861-ea54-4e54-9266-7e033ee168da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.772515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422861-ea54-4e54-9266-7e033ee168da admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.772962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422861-ea54-4e54-9266-7e033ee168da admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.773347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422861-ea54-4e54-9266-7e033ee168da admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.774926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97422861-ea54-4e54-9266-7e033ee168da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.777455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97422861-ea54-4e54-9266-7e033ee168da admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.787773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1105/2162] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:53.798441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f185e39-0dbf-4a2d-bdc2-19126fc8d031 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.798441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f185e39-0dbf-4a2d-bdc2-19126fc8d031 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.798441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f185e39-0dbf-4a2d-bdc2-19126fc8d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.820433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3f185e39-0dbf-4a2d-bdc2-19126fc8d031 None None] MFA Rules not processed for user `1c161ed8c0814110b212bbfd32d62a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:53.821668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3f185e39-0dbf-4a2d-bdc2-19126fc8d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:53.832108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=6,keystone:SELECT=87,keystone:INSERT=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:53.876403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1058/2163] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:53.876403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.888412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5179c-ecdb-4255-a61e-f4e8c39ed386 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.892734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1106/2164] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:53.893612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.893759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f185e39-0dbf-4a2d-bdc2-19126fc8d031 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.894467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.895093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.895422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.896284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.898048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.904540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f185e39-0dbf-4a2d-bdc2-19126fc8d031 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.908617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1059/2165] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:53.909386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.909552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.910190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.910474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.910771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.911077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:53.912073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.913682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.924084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1107/2166] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:53.924858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:53.935305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2abfe-0775-4f42-975f-6a9c8ce9de4b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.939907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.942175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.942175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.942175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] PATH_INFO: `/v3/projects/a0d880114866423480268022d7dd20c5/users/906ca078a25447b3b9c01f4fc76e7090/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.961554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d880114866423480268022d7dd20c5, user_id=906ca078a25447b3b9c01f4fc76e7090, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.963237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.973368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe1cc2-85ed-4409-a02b-b3a177a36b60 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:53.977701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:53.978340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:53.978619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:53.978890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:53.987876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:53.989550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:53.999386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1060/2167] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:54.000160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:54.013141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf7f4c-8e80-412c-85d6-e0df4263fcbc service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:54.017018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:54.017624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.017901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.018199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.019014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:54.020720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:54.031996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1061/2168] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:54.036098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:54.043228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bb286-5e3a-443e-bf0c-d107e55e9930 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:54.047810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:54.048409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.048686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.048958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] PATH_INFO: `/v3/users/1c161ed8c0814110b212bbfd32d62a85` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.050132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c161ed8c0814110b212bbfd32d62a85)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:54.052055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:54.060612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1108/2169] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:53 2022] PUT /identity/v3/projects/a0d880114866423480268022d7dd20c5/users/906ca078a25447b3b9c01f4fc76e7090/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:54.061495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:54.124283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908c949-91f8-4ca5-a140-4bfc6f359c57 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:54.128969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d80b44-fd7e-4b98-af43-9aa4387206d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:54.129629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80b44-fd7e-4b98-af43-9aa4387206d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.129928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80b44-fd7e-4b98-af43-9aa4387206d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.130212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d80b44-fd7e-4b98-af43-9aa4387206d9 admin admin] PATH_INFO: `/v3/projects/52bd08da2d1746c6a84cf75143ae6bf6/users/789e8d5e223a42379153011a7b986d50/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.145128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d80b44-fd7e-4b98-af43-9aa4387206d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52bd08da2d1746c6a84cf75143ae6bf6, user_id=789e8d5e223a42379153011a7b986d50, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:54.148073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d80b44-fd7e-4b98-af43-9aa4387206d9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:54.154054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:54.251937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c161ed8c0814110b212bbfd32d62a85'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:54.254683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1109/2170] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:54 2022] PUT /identity/v3/projects/52bd08da2d1746c6a84cf75143ae6bf6/users/789e8d5e223a42379153011a7b986d50/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:54.256665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda9b5f-3ef5-4b92-bc30-d6d342826c35 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.258938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda9b5f-3ef5-4b92-bc30-d6d342826c35 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.258938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda9b5f-3ef5-4b92-bc30-d6d342826c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.261234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:54.267366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c161ed8c0814110b212bbfd32d62a85'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:54.279324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-acda9b5f-3ef5-4b92-bc30-d6d342826c35 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:54.280524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-acda9b5f-3ef5-4b92-bc30-d6d342826c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:54.338191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1110/2171] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:54.341510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.341510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.341510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.360298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:54.361522 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:54.363155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c161ed8c0814110b212bbfd32d62a85'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:54.370446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1111/2172] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:54.378021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:54.378288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c161ed8c0814110b212bbfd32d62a85'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:54.378288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9c4f4860-ee00-4cf5-9713-f034c61299da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c161ed8c0814110b212bbfd32d62a85'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:54.378441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:54.378514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1062/2173] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:40:54 2022] DELETE /identity/v3/users/1c161ed8c0814110b212bbfd32d62a85 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:54.383369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.383369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.383608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.402951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:54.404159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:54.426288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:54.448682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d292b05-8346-48d3-9e72-139f5c3d6a09 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:54.465980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1063/2174] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:40:54.466951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:54.468219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:54.504316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:54.504867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.505267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.505588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.507445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:54.519054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:54.520483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:54.530340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae1b0a-1ea4-4589-a021-3a611c1f29b3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:54.530501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1112/2175] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:54.534160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:54.534823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.535101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.535371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.535638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:54.537691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:54.543708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:54.552781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7943b-1e0c-4e9d-8e75-8f8112ed35b5 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:54.553568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:54.559602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:54.561184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086742a8-b10d-4ef1-b6b5-f0c2530915ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:54.561184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086742a8-b10d-4ef1-b6b5-f0c2530915ad service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:54.561184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086742a8-b10d-4ef1-b6b5-f0c2530915ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:54.561184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086742a8-b10d-4ef1-b6b5-f0c2530915ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:54.569863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086742a8-b10d-4ef1-b6b5-f0c2530915ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:54.571341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086742a8-b10d-4ef1-b6b5-f0c2530915ad service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:54.571888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1064/2176] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:54.581707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1113/2177] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:40:55.163574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:55.165081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:55.177578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560adde-0178-4dd9-bf4e-9c448147734e service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:55.182666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:55.183417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:55.183808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:55.184181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] PATH_INFO: `/v3/projects/5ce49bc8030e4508a0f3b4bdf6d49ecd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:55.186112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce49bc8030e4508a0f3b4bdf6d49ecd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:55.187687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:55.324692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce49bc8030e4508a0f3b4bdf6d49ecd', 'request_id': 'req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:55.334934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce49bc8030e4508a0f3b4bdf6d49ecd', 'request_id': 'req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:55.343430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce49bc8030e4508a0f3b4bdf6d49ecd', 'request_id': 'req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:55.456831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:55.460644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] The token cache is being invalidate because project 5ce49bc8030e4508a0f3b4bdf6d49ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:40:55.460932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:55.463747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ff5bb4ef-8ac4-4de8-8813-842ae41d86e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:40:55.463747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1065/2178] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:40:55 2022] DELETE /identity/v3/projects/5ce49bc8030e4508a0f3b4bdf6d49ecd => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:55.468347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35051230-645f-4218-a5f2-ac5d98d2e65f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:55.468778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35051230-645f-4218-a5f2-ac5d98d2e65f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:55.468890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35051230-645f-4218-a5f2-ac5d98d2e65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:55.488390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-35051230-645f-4218-a5f2-ac5d98d2e65f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:55.489589 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-35051230-645f-4218-a5f2-ac5d98d2e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:55.564472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1114/2179] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:55.566366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341edef-cace-4904-8926-61a4cbe4c248 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:55.567184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341edef-cace-4904-8926-61a4cbe4c248 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:55.567466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341edef-cace-4904-8926-61a4cbe4c248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:55.589706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5341edef-cace-4904-8926-61a4cbe4c248 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:55.591910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5341edef-cace-4904-8926-61a4cbe4c248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:55.603725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1066/2180] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:55.605167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35051230-645f-4218-a5f2-ac5d98d2e65f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:55.616219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35051230-645f-4218-a5f2-ac5d98d2e65f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:55.620967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:55.621585 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:55.621868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:55.622164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:55.622881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:40:55.623187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:55.624800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:55.640810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1115/2181] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:40:55.642730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341edef-cace-4904-8926-61a4cbe4c248 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:55.652647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341edef-cace-4904-8926-61a4cbe4c248 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:55.657548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:55.658180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:55.658475 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:55.658838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:55.659567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:55.661423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:55.757139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1067/2182] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:40:55 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:40:55.758366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:55.768894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e8ccb-4353-4492-b89e-21bee294afb6 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:55.825761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:55.826444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:55.826729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:55.826998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:55.827671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:55.829623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:55.912684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:40:55.936497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1116/2183] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:40:55 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:40:55.936708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:55.950692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd03ce9-3cce-4941-a6a1-5a2626be02a2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:55.955955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:55.956557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:55.956836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:55.957107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:55.957852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:55.960064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:55.971263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1068/2184] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:55.983385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:55.993894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89201c59-022e-452b-98b9-884edd582037 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:55.999933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616759a-1700-4a3a-8766-e91d68a42a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:56.000916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616759a-1700-4a3a-8766-e91d68a42a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:56.001350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616759a-1700-4a3a-8766-e91d68a42a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:56.001715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616759a-1700-4a3a-8766-e91d68a42a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:56.002677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616759a-1700-4a3a-8766-e91d68a42a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:56.004812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616759a-1700-4a3a-8766-e91d68a42a66 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:56.018349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1117/2185] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:40:56.023282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:56.034441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee912c-0f8f-4b8a-801b-cf143682a97f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:56.039080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:56.039734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:56.040009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:56.040384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] PATH_INFO: `/v3/projects/25e1ffd98adb4875840074ccb3153e8d/users/48418630b4b04f70aaf0ebb901e4f49f/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:56.055616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e1ffd98adb4875840074ccb3153e8d, user_id=48418630b4b04f70aaf0ebb901e4f49f, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:56.058276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:56.201942 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1069/2186] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:40:56 2022] PUT /identity/v3/projects/25e1ffd98adb4875840074ccb3153e8d/users/48418630b4b04f70aaf0ebb901e4f49f/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:40:56.212445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59d790-47f6-4d5e-afdd-54a8fa105e80 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:56.212734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59d790-47f6-4d5e-afdd-54a8fa105e80 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:56.213012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59d790-47f6-4d5e-afdd-54a8fa105e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:56.240699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2c59d790-47f6-4d5e-afdd-54a8fa105e80 None None] MFA Rules not processed for user `48418630b4b04f70aaf0ebb901e4f49f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:40:56.241813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2c59d790-47f6-4d5e-afdd-54a8fa105e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:56.307043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1118/2187] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:40:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:40:56.322321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:40:56.325566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:40:56.398809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac96ca6-7b40-4fd8-9b8a-c636aab5135f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:40:56.402473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fc508-7a1a-430c-a5cd-6cabb188e45c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:40:56.403128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fc508-7a1a-430c-a5cd-6cabb188e45c service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:40:56.403405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fc508-7a1a-430c-a5cd-6cabb188e45c service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:40:56.403675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fc508-7a1a-430c-a5cd-6cabb188e45c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:40:56.414129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fc508-7a1a-430c-a5cd-6cabb188e45c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:40:56.415897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fc508-7a1a-430c-a5cd-6cabb188e45c service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:40:56.441645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1070/2188] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:40:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:01.023107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59d790-47f6-4d5e-afdd-54a8fa105e80 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:01.191138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59d790-47f6-4d5e-afdd-54a8fa105e80 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:01.206455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:01.207129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:01.207407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:01.207686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] PATH_INFO: `/v3/users/48418630b4b04f70aaf0ebb901e4f49f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:01.209207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48418630b4b04f70aaf0ebb901e4f49f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:01.211204 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:01.443903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48418630b4b04f70aaf0ebb901e4f49f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:01.463182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48418630b4b04f70aaf0ebb901e4f49f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:01.540793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48418630b4b04f70aaf0ebb901e4f49f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:01.542255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48418630b4b04f70aaf0ebb901e4f49f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:01.543233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48418630b4b04f70aaf0ebb901e4f49f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:01.552666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:01.576122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1119/2189] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:41:01 2022] DELETE /identity/v3/users/48418630b4b04f70aaf0ebb901e4f49f => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:41:01.581106 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b5350-0a8b-4161-841f-206691c05e6c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:01.581975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b5350-0a8b-4161-841f-206691c05e6c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:01.582463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b5350-0a8b-4161-841f-206691c05e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:01.616926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2f9b5350-0a8b-4161-841f-206691c05e6c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:01.618187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2f9b5350-0a8b-4161-841f-206691c05e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:01.780308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1071/2190] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:01.807405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:01.810942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:01.970848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f138e1d-859c-4cb6-9e67-6cee89163962 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:01.973386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:01.979990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:01.980543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:01.981133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:01.995023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:01.998416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:02.035153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1120/2191] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:02.191044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c6029-446f-4f87-9847-1c8876a64fb6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.191436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c6029-446f-4f87-9847-1c8876a64fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.191817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c6029-446f-4f87-9847-1c8876a64fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.244595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-066c6029-446f-4f87-9847-1c8876a64fb6 None None] MFA Rules not processed for user `789e8d5e223a42379153011a7b986d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:02.247420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-066c6029-446f-4f87-9847-1c8876a64fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:02.288395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:02.351049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1072/2192] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:02.355139 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:02.358497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:02.388068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454231bc-9faa-4fbb-b364-1ba87eafade3 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:02.393779 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:02.394469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.394746 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.402443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.402443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:02.402443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:02.408113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2c62d-016c-4022-b313-5b7dfe07bc88 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.408529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2c62d-016c-4022-b313-5b7dfe07bc88 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.408919 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2c62d-016c-4022-b313-5b7dfe07bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.449403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-28f2c62d-016c-4022-b313-5b7dfe07bc88 None None] MFA Rules not processed for user `906ca078a25447b3b9c01f4fc76e7090`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:02.450847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-28f2c62d-016c-4022-b313-5b7dfe07bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:02.529735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1073/2193] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:02.543771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.544060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.544330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.560545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1121/2194] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:41:02 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:41:02.561935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:02.603558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] MFA Rules not processed for user `906ca078a25447b3b9c01f4fc76e7090`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:02.607098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:02.665900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aa28a-57a5-4322-8263-d8fb35187354 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:02.671492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:02.672114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.672395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.672675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.673331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:02.688237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:02.716432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1074/2195] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:41:02.745345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:02.748645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:02.868881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1122/2196] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:41:02 2022] POST /identity/v3/users => generated 299 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:41:02.870045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:02.875482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:02.883648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ce06f-6f19-46cf-a4f2-5a6104e5412c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:02.884655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1c4d-ee07-4ee1-9d3e-4d0fe60edbf6 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:02.887474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:02.888096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.888378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.888654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.889818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:02.890486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.890759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.891084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.891737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:02.893355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:02.899019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:02.901100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:02.913348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1075/2197] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:41:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:02.914322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:02.915917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:02.916482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1123/2198] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:41:02.920602 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:02.931358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e99f2-c307-4bac-b05f-0e21b775a8ac service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:02.941749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:02.941749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.941749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.941749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] PATH_INFO: `/v3/projects/25e1ffd98adb4875840074ccb3153e8d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.942430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a42f-3042-4006-a374-f140d59a1d8b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:02.942569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e1ffd98adb4875840074ccb3153e8d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:02.942569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:02.945483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:02.946826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:02.947372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:02.947836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] PATH_INFO: `/v3/projects/df89794fec88414e88f7a4b8ecbce47f/users/e4030d45acc64a9686ac130a4a372d36/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:02.972513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df89794fec88414e88f7a4b8ecbce47f, user_id=e4030d45acc64a9686ac130a4a372d36, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:02.976971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:03.104000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1124/2199] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:41:02 2022] PUT /identity/v3/projects/df89794fec88414e88f7a4b8ecbce47f/users/e4030d45acc64a9686ac130a4a372d36/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:41:03.107846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:03.120718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e1ffd98adb4875840074ccb3153e8d', 'request_id': 'req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:03.138593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e1ffd98adb4875840074ccb3153e8d', 'request_id': 'req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:03.147337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e1ffd98adb4875840074ccb3153e8d', 'request_id': 'req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:03.161654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:03.199008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47093-cc02-4a50-ada3-5ae9c1fc6640 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:03.250008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] The token cache is being invalidate because project 25e1ffd98adb4875840074ccb3153e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:41:03.250328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:03.252188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:03.266767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1076/2200] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:41:02 2022] DELETE /identity/v3/projects/25e1ffd98adb4875840074ccb3153e8d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:41:03.277799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13320e-3ab9-420d-b881-670c4c6f6423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:03.278586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13320e-3ab9-420d-b881-670c4c6f6423 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:03.278971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13320e-3ab9-420d-b881-670c4c6f6423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:03.279328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13320e-3ab9-420d-b881-670c4c6f6423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:03.280203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13320e-3ab9-420d-b881-670c4c6f6423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:03.282108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13320e-3ab9-420d-b881-670c4c6f6423 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:03.296848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1125/2201] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:41:03.300831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:03.304271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:03.326703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f10447-e3cc-4ed3-b01a-6a800c308d1d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:03.333903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a82826-afc2-4a73-a899-8b0a8eb25bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:03.338369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82826-afc2-4a73-a899-8b0a8eb25bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:03.338369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82826-afc2-4a73-a899-8b0a8eb25bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:03.338695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82826-afc2-4a73-a899-8b0a8eb25bfb admin admin] PATH_INFO: `/v3/projects/df89794fec88414e88f7a4b8ecbce47f/users/e4030d45acc64a9686ac130a4a372d36/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:03.341926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a82826-afc2-4a73-a899-8b0a8eb25bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df89794fec88414e88f7a4b8ecbce47f, user_id=e4030d45acc64a9686ac130a4a372d36, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:03.345419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a82826-afc2-4a73-a899-8b0a8eb25bfb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:03.378995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1077/2202] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:41:03 2022] PUT /identity/v3/projects/df89794fec88414e88f7a4b8ecbce47f/users/e4030d45acc64a9686ac130a4a372d36/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:41:04.361285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:04.361799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:04.362013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:04.387425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] MFA Rules not processed for user `906ca078a25447b3b9c01f4fc76e7090`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:04.388772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:04.466391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1126/2203] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:11.780281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafb686-49b1-4a22-96f7-8c487a531777 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:11.780738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafb686-49b1-4a22-96f7-8c487a531777 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:11.780847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafb686-49b1-4a22-96f7-8c487a531777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:11.820735 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ebafb686-49b1-4a22-96f7-8c487a531777 None None] MFA Rules not processed for user `e4030d45acc64a9686ac130a4a372d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:11.821979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ebafb686-49b1-4a22-96f7-8c487a531777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:11.913499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1078/2204] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:11.925735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:11.927823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:11.996531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea74743-accb-4512-9ce2-7b41f51c8b19 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:12.002160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:12.002851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:12.003124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:12.003397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:12.005473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:12.040237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:12.064923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:12.066400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:12.075578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1127/2205] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:41:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:12.344522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:12.344814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:12.348560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:12.365379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:12.366628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:12.411108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:12.426991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1079/2206] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:41:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:12.471811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:12.473691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:12.569945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58649ade-00ed-4667-83bc-a1f44db09c2c service glance] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:12.577846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13e66f-c8e7-420d-987b-231c2cff6724 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:12.578533 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13e66f-c8e7-420d-987b-231c2cff6724 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:12.578818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13e66f-c8e7-420d-987b-231c2cff6724 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:12.579089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13e66f-c8e7-420d-987b-231c2cff6724 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:12.596708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13e66f-c8e7-420d-987b-231c2cff6724 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:12.596708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13e66f-c8e7-420d-987b-231c2cff6724 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:12.603870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1128/2207] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:41:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:41:21.482888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:21.485183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:21.589554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87461f8f-8e35-4409-8f43-baff6251bad4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:21.597820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:21.599041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:21.599440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:21.599810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] PATH_INFO: `/v3/users/383f0bd30d0249938d35fa622177c457` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:21.601612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=383f0bd30d0249938d35fa622177c457)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:21.604100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:21.855371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '383f0bd30d0249938d35fa622177c457'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:21.880262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '383f0bd30d0249938d35fa622177c457'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:21.951235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '383f0bd30d0249938d35fa622177c457'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:21.961406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '383f0bd30d0249938d35fa622177c457'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:21.962291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '383f0bd30d0249938d35fa622177c457'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:21.968984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1080/2208] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:41:21 2022] DELETE /identity/v3/users/383f0bd30d0249938d35fa622177c457 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:41:21.974228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:21.975256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:21.975561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:21.997430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:21.999167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:22.059635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:22.061492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:22.085786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1129/2209] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:22.103711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:22.103711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:22.163514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712331d1-f73b-429b-8728-343ee0c997cf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:22.169022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:22.169827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:22.170117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:22.170740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] PATH_INFO: `/v3/users/972ea719298f458a90d289eef6c24bb5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:22.173627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=972ea719298f458a90d289eef6c24bb5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:22.176769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:22.181265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a36be-c48e-47df-bc70-076fb69f8dfd None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:22.186459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526318e-8dd2-45ff-b5f0-3d0b045b7cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:22.187171 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526318e-8dd2-45ff-b5f0-3d0b045b7cee service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:22.187526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526318e-8dd2-45ff-b5f0-3d0b045b7cee service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:22.187877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526318e-8dd2-45ff-b5f0-3d0b045b7cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:22.192673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:22.200775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526318e-8dd2-45ff-b5f0-3d0b045b7cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:22.202270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526318e-8dd2-45ff-b5f0-3d0b045b7cee service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:22.213951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1130/2210] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:22.361281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972ea719298f458a90d289eef6c24bb5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:22.399292 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972ea719298f458a90d289eef6c24bb5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:22.498522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972ea719298f458a90d289eef6c24bb5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:22.508438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972ea719298f458a90d289eef6c24bb5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:22.509249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972ea719298f458a90d289eef6c24bb5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:22.516647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1081/2211] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:41:22 2022] DELETE /identity/v3/users/972ea719298f458a90d289eef6c24bb5 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:41:22.523836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:22.524127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:22.524407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:22.550963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:22.552889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:22.554786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:22.682754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1131/2212] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:22.701273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:22.702739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:22.864146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282406e6-4529-4416-b01e-37dfb16973d2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:22.864146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:22.864146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:22.864146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:22.864867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:22.864867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:22.864867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:22.864867 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1082/2213] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:23.398827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:23.400428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:23.416292 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf27eb-3150-4171-9b67-fbae36d4edc5 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:23.425199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:23.425199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:23.425199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:23.425199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] PATH_INFO: `/v3/projects/3bc58d1e7ca44ed0b776999dd09beb0a` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:23.425622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc58d1e7ca44ed0b776999dd09beb0a)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:23.427649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:23.621523 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc58d1e7ca44ed0b776999dd09beb0a', 'request_id': 'req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:23.631398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc58d1e7ca44ed0b776999dd09beb0a', 'request_id': 'req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:23.714919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc58d1e7ca44ed0b776999dd09beb0a', 'request_id': 'req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:23.735578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] The token cache is being invalidate because project 3bc58d1e7ca44ed0b776999dd09beb0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:41:23.735872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:23.740872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-83bef9b8-49b1-4b4b-bef8-59073a3eaea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:23.764913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1132/2214] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:41:23 2022] DELETE /identity/v3/projects/3bc58d1e7ca44ed0b776999dd09beb0a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:41:24.046434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:24.048209 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:24.075138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b4513-e4c9-465c-9f27-83b67ed90399 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:24.195604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:24.196208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:24.196487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:24.196759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] PATH_INFO: `/v3/projects/13d415416e294240b718f4886f462da4` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:24.197932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d415416e294240b718f4886f462da4)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:24.200081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:24.453757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:24.466410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d415416e294240b718f4886f462da4', 'request_id': 'req-afe08048-220e-42a7-b321-9deda64fcdd8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:24.472198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d415416e294240b718f4886f462da4', 'request_id': 'req-afe08048-220e-42a7-b321-9deda64fcdd8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:24.479510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d415416e294240b718f4886f462da4', 'request_id': 'req-afe08048-220e-42a7-b321-9deda64fcdd8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:24.561509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] The token cache is being invalidate because project 13d415416e294240b718f4886f462da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:41:24.561801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:24.562438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afe08048-220e-42a7-b321-9deda64fcdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:24.564020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1083/2215] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:41:24 2022] DELETE /identity/v3/projects/13d415416e294240b718f4886f462da4 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:41:24.569441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc7061-01da-466f-bbb6-12876e690a07 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:24.569931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc7061-01da-466f-bbb6-12876e690a07 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:24.570130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc7061-01da-466f-bbb6-12876e690a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:24.590013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-99bc7061-01da-466f-bbb6-12876e690a07 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:24.591333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-99bc7061-01da-466f-bbb6-12876e690a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:24.656939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1133/2216] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:24.665356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5b5a-5c0d-413c-be73-628713506f14 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:24.665644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5b5a-5c0d-413c-be73-628713506f14 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:24.665920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5b5a-5c0d-413c-be73-628713506f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:24.686280 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6ffc5b5a-5c0d-413c-be73-628713506f14 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:24.687486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6ffc5b5a-5c0d-413c-be73-628713506f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:24.695983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1084/2217] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:24.700336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc7061-01da-466f-bbb6-12876e690a07 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:24.710494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc7061-01da-466f-bbb6-12876e690a07 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:24.714954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:24.715550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:24.715823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:24.716092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:24.716818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:41:24.717117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:24.718759 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:24.730108 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1134/2218] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:41:24.734329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc5b5a-5c0d-413c-be73-628713506f14 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:24.744363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc5b5a-5c0d-413c-be73-628713506f14 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:24.749404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad6353-26e1-4c3f-911a-f272bbe94627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:24.750020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad6353-26e1-4c3f-911a-f272bbe94627 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:24.750327 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad6353-26e1-4c3f-911a-f272bbe94627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:24.754233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad6353-26e1-4c3f-911a-f272bbe94627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:24.754233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad6353-26e1-4c3f-911a-f272bbe94627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:24.754233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad6353-26e1-4c3f-911a-f272bbe94627 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:24.860414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1085/2219] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:41:24 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:41:24.865151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:24.875271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f974a8-ccf7-4f65-88d2-a2f53f94db0f admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:24.888833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:24.944395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:24.944395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:24.944395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:24.944395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:24.944920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:24.944920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:24.990354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc427f40-2264-4d6a-8004-346c31106719 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:24.990636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc427f40-2264-4d6a-8004-346c31106719 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:24.990907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc427f40-2264-4d6a-8004-346c31106719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.009715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-dc427f40-2264-4d6a-8004-346c31106719 None None] MFA Rules not processed for user `789e8d5e223a42379153011a7b986d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:25.010950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-dc427f40-2264-4d6a-8004-346c31106719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:25.025724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1135/2220] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:41:24 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:41:25.029350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:25.040159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73475c-9c71-40ad-ab2a-937f2a99790d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:25.044892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:25.045491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:25.045839 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:25.046086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.046803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:25.048429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:25.064128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1136/2221] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:41:25.065102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:25.069079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1086/2222] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:41:25.075223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85e04-b393-4ca0-8ebe-4aff821a6120 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:25.080039 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95d00b-e130-41d5-b3d3-fcb962efb278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:25.080859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d00b-e130-41d5-b3d3-fcb962efb278 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:25.081138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d00b-e130-41d5-b3d3-fcb962efb278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:25.081445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d00b-e130-41d5-b3d3-fcb962efb278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.082261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95d00b-e130-41d5-b3d3-fcb962efb278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:25.083829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95d00b-e130-41d5-b3d3-fcb962efb278 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:25.097028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1137/2223] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:41:25.098961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc427f40-2264-4d6a-8004-346c31106719 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:25.109900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc427f40-2264-4d6a-8004-346c31106719 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:25.115476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:25.116150 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:25.116434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:25.116712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] PATH_INFO: `/v3/projects/0b155f32b1bb4782a78c1b799da2e6ef/users/ab7d6d09a10f41a7bfbba92e809d5d69/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.131900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b155f32b1bb4782a78c1b799da2e6ef, user_id=ab7d6d09a10f41a7bfbba92e809d5d69, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:25.134672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:25.227127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1087/2224] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:41:25 2022] PUT /identity/v3/projects/0b155f32b1bb4782a78c1b799da2e6ef/users/ab7d6d09a10f41a7bfbba92e809d5d69/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:41:25.237118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe194c-f1fe-42db-b8c0-36d4792ee1ef None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:25.237410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe194c-f1fe-42db-b8c0-36d4792ee1ef None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:25.238513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe194c-f1fe-42db-b8c0-36d4792ee1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.274083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3bbe194c-f1fe-42db-b8c0-36d4792ee1ef None None] MFA Rules not processed for user `ab7d6d09a10f41a7bfbba92e809d5d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:25.275620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3bbe194c-f1fe-42db-b8c0-36d4792ee1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:25.345193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1138/2225] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:25.357877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:25.359918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:25.427081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca47465-da0f-42f4-af33-589d297baf4f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:25.431753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a1cd7-e823-43da-b023-1cfc5498025a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:25.434675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a1cd7-e823-43da-b023-1cfc5498025a service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:25.434675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a1cd7-e823-43da-b023-1cfc5498025a service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:25.434675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a1cd7-e823-43da-b023-1cfc5498025a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.444460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a1cd7-e823-43da-b023-1cfc5498025a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:25.446085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a1cd7-e823-43da-b023-1cfc5498025a service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:25.473803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1088/2226] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:41:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:25.494329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe194c-f1fe-42db-b8c0-36d4792ee1ef None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:25.591049 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe194c-f1fe-42db-b8c0-36d4792ee1ef None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:25.596907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:25.597971 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:25.598275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:25.598548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] PATH_INFO: `/v3/users/ab7d6d09a10f41a7bfbba92e809d5d69` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.599769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7d6d09a10f41a7bfbba92e809d5d69)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:25.601868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:25.621954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:25.781970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7d6d09a10f41a7bfbba92e809d5d69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:25.795555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7d6d09a10f41a7bfbba92e809d5d69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:25.864004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7d6d09a10f41a7bfbba92e809d5d69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:25.865371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7d6d09a10f41a7bfbba92e809d5d69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:25.866112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7d6d09a10f41a7bfbba92e809d5d69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:25.889348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1139/2227] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:41:25 2022] DELETE /identity/v3/users/ab7d6d09a10f41a7bfbba92e809d5d69 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:41:25.895195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:25.895673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:25.895956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:25.917530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:25.919019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:25.930297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:25.980608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1089/2228] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:25.990851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:25.992296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:26.051890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc4c72-4602-43ee-a3e4-71934a7e7d5d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:26.055918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219adfb-fec0-4aea-aa5f-2f2dbfa6ce6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:26.056595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219adfb-fec0-4aea-aa5f-2f2dbfa6ce6b service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:26.056880 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219adfb-fec0-4aea-aa5f-2f2dbfa6ce6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:26.057156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219adfb-fec0-4aea-aa5f-2f2dbfa6ce6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:26.071446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219adfb-fec0-4aea-aa5f-2f2dbfa6ce6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:26.073009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219adfb-fec0-4aea-aa5f-2f2dbfa6ce6b service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:26.088196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1140/2229] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:41:26.487839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:26.489491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:26.501481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e12ab8-a73f-4810-9b54-c75986d81f7c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:26.507208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:26.512688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:26.512688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:26.512688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] PATH_INFO: `/v3/projects/0b155f32b1bb4782a78c1b799da2e6ef` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:26.512688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b155f32b1bb4782a78c1b799da2e6ef)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:26.512688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:26.636598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b155f32b1bb4782a78c1b799da2e6ef', 'request_id': 'req-59608270-bc2c-4cbd-ab99-b2976cd1c375'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:26.642158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b155f32b1bb4782a78c1b799da2e6ef', 'request_id': 'req-59608270-bc2c-4cbd-ab99-b2976cd1c375'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:26.648371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b155f32b1bb4782a78c1b799da2e6ef', 'request_id': 'req-59608270-bc2c-4cbd-ab99-b2976cd1c375'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:26.762833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] The token cache is being invalidate because project 0b155f32b1bb4782a78c1b799da2e6ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:41:26.763373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:26.764173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:41:26.766163 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1090/2230] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:41:26 2022] DELETE /identity/v3/projects/0b155f32b1bb4782a78c1b799da2e6ef => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:41:30.001395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:30.001878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:30.001982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:30.021920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:41:30.023190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:30.080137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1141/2231] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:41:30.092313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:41:30.093776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:41:30.177573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59608270-bc2c-4cbd-ab99-b2976cd1c375 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:41:30.185531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92854c46-130d-4859-be62-622f5ffe4dd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:41:30.186220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92854c46-130d-4859-be62-622f5ffe4dd9 service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:41:30.186473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92854c46-130d-4859-be62-622f5ffe4dd9 service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:41:30.186749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92854c46-130d-4859-be62-622f5ffe4dd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:41:30.197267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92854c46-130d-4859-be62-622f5ffe4dd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:41:30.199062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92854c46-130d-4859-be62-622f5ffe4dd9 service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:41:30.208840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1091/2232] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:41:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:41:40.073097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=9,keystone:SELECT=71,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:41:40.205141 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:42:09.006395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:42:09.008335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:42:09.065575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2431a-8409-4861-8365-929af7d32c54 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:42:09.069036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:42:09.070416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:42:09.070704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:42:09.070979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:42:09.072717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:42:09.135444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:42:09.136886 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:42:09.146397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1142/2233] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:42:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:42:19.142126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=50 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:42:33.852452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee309b6-ebf7-445b-9894-c5fc499463cf None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:42:33.852988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee309b6-ebf7-445b-9894-c5fc499463cf None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:42:33.852988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee309b6-ebf7-445b-9894-c5fc499463cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:42:33.873515 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bee309b6-ebf7-445b-9894-c5fc499463cf None None] MFA Rules not processed for user `906ca078a25447b3b9c01f4fc76e7090`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:42:33.874846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bee309b6-ebf7-445b-9894-c5fc499463cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:42:33.929089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1092/2234] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:42:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:42:35.901733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:42:35.904107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:42:35.974961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fde5-c92e-4143-95c6-9f0a0cd61a11 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:42:35.979962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ef220-1647-48a6-9d3e-02d84c36d23b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:42:35.980568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ef220-1647-48a6-9d3e-02d84c36d23b service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:42:35.980844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ef220-1647-48a6-9d3e-02d84c36d23b service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:42:35.981201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ef220-1647-48a6-9d3e-02d84c36d23b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:42:35.983004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-348ef220-1647-48a6-9d3e-02d84c36d23b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:42:35.994063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ef220-1647-48a6-9d3e-02d84c36d23b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:42:35.995665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ef220-1647-48a6-9d3e-02d84c36d23b service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:42:36.005342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1143/2235] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:42:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:42:40.764329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899cf18-ff80-4834-9c73-fe45b1b91e00 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:42:40.765130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899cf18-ff80-4834-9c73-fe45b1b91e00 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:42:40.765130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899cf18-ff80-4834-9c73-fe45b1b91e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:42:40.785383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d899cf18-ff80-4834-9c73-fe45b1b91e00 None None] MFA Rules not processed for user `c370dd8fa3aa407ca22cce8425fcbc25`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:42:40.786792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d899cf18-ff80-4834-9c73-fe45b1b91e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:42:40.844588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1093/2236] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:42:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:42:46.001307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=32 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:42:50.837437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=50 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:42:51.936955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066cf5a-2c46-4b4b-aa2e-36a44d1b9316 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:42:51.940689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066cf5a-2c46-4b4b-aa2e-36a44d1b9316 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:42:51.940689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066cf5a-2c46-4b4b-aa2e-36a44d1b9316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:42:51.961359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b066cf5a-2c46-4b4b-aa2e-36a44d1b9316 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:42:51.962699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b066cf5a-2c46-4b4b-aa2e-36a44d1b9316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:42:51.972280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1144/2237] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:42:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:42:52.015420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899cf18-ff80-4834-9c73-fe45b1b91e00 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:42:52.036642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899cf18-ff80-4834-9c73-fe45b1b91e00 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:42:52.041760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128307e-7bb5-49de-8acf-ba67054a66ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:42:52.042489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128307e-7bb5-49de-8acf-ba67054a66ca service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:42:52.042771 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128307e-7bb5-49de-8acf-ba67054a66ca service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:42:52.043312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128307e-7bb5-49de-8acf-ba67054a66ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:42:52.052422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128307e-7bb5-49de-8acf-ba67054a66ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:42:52.054200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128307e-7bb5-49de-8acf-ba67054a66ca service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:42:52.062543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1094/2238] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:42:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:43:01.955417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:43:02.059221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:43:09.861645 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995015ea-d473-446a-b515-88e49f7958be None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:09.862138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995015ea-d473-446a-b515-88e49f7958be None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:09.862251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995015ea-d473-446a-b515-88e49f7958be None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:09.863591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1145/2239] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:43:09.869507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de641c8-97ab-4a06-971b-8cb55b764e68 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:09.870037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de641c8-97ab-4a06-971b-8cb55b764e68 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:09.870037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de641c8-97ab-4a06-971b-8cb55b764e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:09.890019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2de641c8-97ab-4a06-971b-8cb55b764e68 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:43:09.891284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2de641c8-97ab-4a06-971b-8cb55b764e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:43:09.963551 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1095/2240] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:43:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:43:09.976439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995015ea-d473-446a-b515-88e49f7958be None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:43:09.979480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-995015ea-d473-446a-b515-88e49f7958be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:43:10.041345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995015ea-d473-446a-b515-88e49f7958be None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:43:10.044892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80039e-d0ab-4ee7-b5ff-1b510f82436f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:43:10.045505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80039e-d0ab-4ee7-b5ff-1b510f82436f service glance] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:10.045777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80039e-d0ab-4ee7-b5ff-1b510f82436f service glance] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:10.046088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80039e-d0ab-4ee7-b5ff-1b510f82436f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:10.058355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80039e-d0ab-4ee7-b5ff-1b510f82436f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:43:10.059815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80039e-d0ab-4ee7-b5ff-1b510f82436f service glance] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:43:10.070612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1146/2241] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:43:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:43:10.255358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244d6b-3336-42a0-90ea-b1e7b0c8d4e6 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:10.255681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244d6b-3336-42a0-90ea-b1e7b0c8d4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:10.255928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a244d6b-3336-42a0-90ea-b1e7b0c8d4e6 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:10.257290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1096/2242] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:43:10.261851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f760c-07ff-4751-a452-d0db8027d09d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:10.262152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f760c-07ff-4751-a452-d0db8027d09d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:10.263793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f760c-07ff-4751-a452-d0db8027d09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:10.283013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ed3f760c-07ff-4751-a452-d0db8027d09d None None] MFA Rules not processed for user `d5c308c33c674f1ea7111c0297174c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:43:10.286889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ed3f760c-07ff-4751-a452-d0db8027d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:43:10.291140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1147/2243] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:43:10.302974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244d6b-3336-42a0-90ea-b1e7b0c8d4e6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:43:10.314185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a244d6b-3336-42a0-90ea-b1e7b0c8d4e6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:43:10.317697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b143b38a-3256-4631-b194-afee7ea3d0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:43:10.318322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143b38a-3256-4631-b194-afee7ea3d0aa service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:10.318603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143b38a-3256-4631-b194-afee7ea3d0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:10.318876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b143b38a-3256-4631-b194-afee7ea3d0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:10.327205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143b38a-3256-4631-b194-afee7ea3d0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:43:10.328627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b143b38a-3256-4631-b194-afee7ea3d0aa service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:43:10.337181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1097/2244] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:43:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:43:10.418212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4768a1-f3a1-4ae5-b6f7-b65235cd2907 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:10.418504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4768a1-f3a1-4ae5-b6f7-b65235cd2907 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:10.418782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4768a1-f3a1-4ae5-b6f7-b65235cd2907 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:10.420151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1148/2245] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:43:10.425647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d2db-4dcc-4fe8-b8d5-d4c2205864c1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:10.425937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d2db-4dcc-4fe8-b8d5-d4c2205864c1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:10.426242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d2db-4dcc-4fe8-b8d5-d4c2205864c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:10.445972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9323d2db-4dcc-4fe8-b8d5-d4c2205864c1 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:43:10.447258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9323d2db-4dcc-4fe8-b8d5-d4c2205864c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:43:10.457393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1098/2246] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:43:10.468184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4768a1-f3a1-4ae5-b6f7-b65235cd2907 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:43:10.470567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dd4768a1-f3a1-4ae5-b6f7-b65235cd2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:43:10.483879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4768a1-f3a1-4ae5-b6f7-b65235cd2907 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:43:10.488372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:43:10.488948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:10.489228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:10.489501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:10.499928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:43:10.501697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:43:10.512624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1149/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:43:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:43:14.473255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2953b6e-1530-42a3-9171-c297bc120284 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:14.473726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2953b6e-1530-42a3-9171-c297bc120284 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:14.473817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2953b6e-1530-42a3-9171-c297bc120284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:14.494112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d2953b6e-1530-42a3-9171-c297bc120284 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:43:14.495282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d2953b6e-1530-42a3-9171-c297bc120284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:43:14.502263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1099/2248] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:43:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:43:14.514399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:43:14.526538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52070353-e278-40d3-9acb-0a7bed1a5059 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:43:14.531092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:43:14.531696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:14.531974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:14.532249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:14.541348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:43:14.542963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:43:14.556627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1150/2249] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:43:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 27 08:43:24.488051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=43 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:43:24.547975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=41 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:43:46.437493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:46.438376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:46.438576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:46.459309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:43:46.460629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:43:46.468926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1100/2250] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 27 08:43:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:43:46.482449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:43:46.493911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da994052-2505-47ce-b981-eee9c1deaa0c service swift] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:43:46.499252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:43:46.499857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:43:46.500130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:43:46.500399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:43:46.508849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:43:46.510444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:43:46.518276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1151/2251] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:43:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:43:56.453234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:43:56.515141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:44:27.008240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:27.009779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:27.071308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65761b9-593f-4d67-9beb-eeb70f335a99 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:27.074921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:27.075534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:27.075813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:27.076087 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:27.077844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:27.091739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:27.093191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:27.103811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1101/2252] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:44:30.987584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:30.996932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f66f16-b68b-440e-9f8c-4e85642107a6 service swift] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:31.000569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:31.001162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:31.001439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:31.001709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:31.003491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:31.017234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:31.018743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:31.029974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1152/2253] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:44:32.162413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:32.163947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:32.178334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e040b81-5f34-4918-bc78-d529a46fe32e service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:32.183866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:32.184462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:32.184743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:32.185017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:32.186990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:32.245179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:32.246803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:32.255833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1102/2254] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:44:41.024697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:44:42.251612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=54 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:44:47.259165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:47.270981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606a18f-c8b0-473b-9561-3392cba2c1ba service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:47.276496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:47.277101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:47.277384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:47.277660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:47.279588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:47.291288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:47.293152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:47.302512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1153/2255] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:44:54.587927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:54.605055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2adf4f-f64d-463f-a65a-78fd734d2ec0 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:54.613965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:54.614804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:54.615269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:54.615652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:54.630287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:54.660874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:54.662682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:54.678377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1103/2256] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:44:54.823491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:54.834249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738ecfe-6fde-4dac-b560-b7b6a933bdd8 service nova] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:54.838466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:54.839198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:54.839741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:54.840089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:54.843098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:54.857649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:54.857649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:54.864590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1154/2257] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:44:55.388429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:55.390758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:55.406249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699ec11-6215-4f7a-878c-a3de5b0a8f91 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:55.412506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6965bd-3655-4563-9edd-fcee8b0c7e00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:55.413112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6965bd-3655-4563-9edd-fcee8b0c7e00 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:55.413391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6965bd-3655-4563-9edd-fcee8b0c7e00 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:55.413664 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6965bd-3655-4563-9edd-fcee8b0c7e00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:55.415637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4c6965bd-3655-4563-9edd-fcee8b0c7e00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:55.484559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6965bd-3655-4563-9edd-fcee8b0c7e00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:55.486199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6965bd-3655-4563-9edd-fcee8b0c7e00 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:55.495982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1104/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:44:56.096197 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:44:56.138541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8dc0f-bdcc-47ba-a81e-0ea17f2d22f4 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:44:56.145672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:44:56.146329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:44:56.146612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:44:56.146894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:44:56.148890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:44:56.166055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:44:56.167766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:44:56.181763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1155/2259] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:44:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:05.492084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=34 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:06.175344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=18 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:07.628058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959192a8-fec4-4586-aa6a-a8e2f366dfef None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:07.628527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959192a8-fec4-4586-aa6a-a8e2f366dfef None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:07.628733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959192a8-fec4-4586-aa6a-a8e2f366dfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:07.647745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-959192a8-fec4-4586-aa6a-a8e2f366dfef None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:07.648943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-959192a8-fec4-4586-aa6a-a8e2f366dfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:07.656764 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1105/2260] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 27 08:45:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:07.800329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:07.811944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6029a9-08f5-48f4-a69a-d7d733b580ae service nova] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:07.817762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:07.818564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:07.818842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:07.819113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:07.828641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:07.833009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:07.839271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1156/2261] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:45:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:45:10.217559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805b27-4922-4843-8f39-ff2f6e51a255 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:10.218080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805b27-4922-4843-8f39-ff2f6e51a255 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:10.218080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805b27-4922-4843-8f39-ff2f6e51a255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:10.244689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-aa805b27-4922-4843-8f39-ff2f6e51a255 None None] MFA Rules not processed for user `c370dd8fa3aa407ca22cce8425fcbc25`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:10.246242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-aa805b27-4922-4843-8f39-ff2f6e51a255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:10.254758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1106/2262] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:10.852678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:10.862745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb4fda-eab8-4cbb-8abe-e088fbe65778 service swift] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:10.866433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047e746-b627-4b3a-8b70-a0cef6d55c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:10.867043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047e746-b627-4b3a-8b70-a0cef6d55c7c service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:10.867394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047e746-b627-4b3a-8b70-a0cef6d55c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:10.867666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047e746-b627-4b3a-8b70-a0cef6d55c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:10.869428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f047e746-b627-4b3a-8b70-a0cef6d55c7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:10.879539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f047e746-b627-4b3a-8b70-a0cef6d55c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:10.881301 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f047e746-b627-4b3a-8b70-a0cef6d55c7c service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:10.891833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1157/2263] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:45:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:11.461358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805b27-4922-4843-8f39-ff2f6e51a255 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:11.478054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805b27-4922-4843-8f39-ff2f6e51a255 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:11.484405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:11.485167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:11.485507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:11.485994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:11.488419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:11.502684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:11.504572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:11.516576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1107/2264] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:45:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:12.351815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daea6d-fdad-4f36-bd94-e1a34d15607b None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:12.352298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daea6d-fdad-4f36-bd94-e1a34d15607b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:12.352415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daea6d-fdad-4f36-bd94-e1a34d15607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:12.376515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-10daea6d-fdad-4f36-bd94-e1a34d15607b None None] MFA Rules not processed for user `26128a80933f4be3bed504d7a2d2785f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:12.378062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-10daea6d-fdad-4f36-bd94-e1a34d15607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:12.385876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1158/2265] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:12.890270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:12.900630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaea686-fdb3-419f-a250-f48ca30414b4 service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:12.905611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54433-ee2c-497b-8ef0-ec5114a43851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:12.906534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54433-ee2c-497b-8ef0-ec5114a43851 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:12.907226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54433-ee2c-497b-8ef0-ec5114a43851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:12.907612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54433-ee2c-497b-8ef0-ec5114a43851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:12.909534 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e2f54433-ee2c-497b-8ef0-ec5114a43851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:12.925419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54433-ee2c-497b-8ef0-ec5114a43851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:12.927561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54433-ee2c-497b-8ef0-ec5114a43851 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:12.937129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1108/2266] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:45:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:13.046321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930cc6-2688-4d8b-8279-c564f1004bd6 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:13.046604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930cc6-2688-4d8b-8279-c564f1004bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:13.047179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930cc6-2688-4d8b-8279-c564f1004bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:13.067900 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3b930cc6-2688-4d8b-8279-c564f1004bd6 None None] MFA Rules not processed for user `906ca078a25447b3b9c01f4fc76e7090`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:13.069488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3b930cc6-2688-4d8b-8279-c564f1004bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:13.077592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1159/2267] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:20.633375 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180b98-279c-42e2-88ba-41f50cc3fed5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:20.633906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180b98-279c-42e2-88ba-41f50cc3fed5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:20.633906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43180b98-279c-42e2-88ba-41f50cc3fed5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:20.640888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1109/2268] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:45:20.667463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:20.667974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:20.668066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:20.692153 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] MFA Rules not processed for user `04d05e0dfd474dfdb848b129941532b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:20.693785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:20.704892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1160/2269] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:45:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:20.740296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43180b98-279c-42e2-88ba-41f50cc3fed5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:20.755417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43180b98-279c-42e2-88ba-41f50cc3fed5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:20.760766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:20.763905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:20.763905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:20.763905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:20.771497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:20.772930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:20.790873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1110/2270] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:45:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:20.809308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:20.811609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:20.889420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d1bf-a20f-4d74-9f58-352f39ab44c1 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:20.893992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:20.894659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:20.894938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:20.895210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] PATH_INFO: `/v3/users/789e8d5e223a42379153011a7b986d50` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:20.896448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=789e8d5e223a42379153011a7b986d50)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:20.898460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:21.110097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789e8d5e223a42379153011a7b986d50'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.125516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789e8d5e223a42379153011a7b986d50'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.200347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789e8d5e223a42379153011a7b986d50'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.201671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789e8d5e223a42379153011a7b986d50'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.202331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789e8d5e223a42379153011a7b986d50'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.234377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1161/2271] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:45:20 2022] DELETE /identity/v3/users/789e8d5e223a42379153011a7b986d50 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:45:21.508048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:21.518668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a1bb5-c0b3-4bd8-bdb3-85f1f8728a56 service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:21.587494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:21.588146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:21.588423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:21.588697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] PATH_INFO: `/v3/projects/52bd08da2d1746c6a84cf75143ae6bf6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:21.589856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52bd08da2d1746c6a84cf75143ae6bf6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:21.591966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:21.710384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52bd08da2d1746c6a84cf75143ae6bf6', 'request_id': 'req-37e6b076-d6c4-458e-b024-1a770c66b328'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.716351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52bd08da2d1746c6a84cf75143ae6bf6', 'request_id': 'req-37e6b076-d6c4-458e-b024-1a770c66b328'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.723448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52bd08da2d1746c6a84cf75143ae6bf6', 'request_id': 'req-37e6b076-d6c4-458e-b024-1a770c66b328'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.826549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] The token cache is being invalidate because project 52bd08da2d1746c6a84cf75143ae6bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:45:21.826934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.827608 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:21.829240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1111/2272] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:45:21 2022] DELETE /identity/v3/projects/52bd08da2d1746c6a84cf75143ae6bf6 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:45:21.833470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:21.835021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:21.849794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8c06a-c504-4ca9-873b-22468961aa8e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:21.886584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:21.915489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:21.916144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:21.916421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:21.916697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] PATH_INFO: `/v3/users/e4030d45acc64a9686ac130a4a372d36` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:21.917864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4030d45acc64a9686ac130a4a372d36)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:21.920084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:22.099273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4030d45acc64a9686ac130a4a372d36'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.112373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4030d45acc64a9686ac130a4a372d36'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.214872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4030d45acc64a9686ac130a4a372d36'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.215443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4030d45acc64a9686ac130a4a372d36'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.216043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-025a08df-369c-47c2-9ec0-6f552dadf494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4030d45acc64a9686ac130a4a372d36'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.229761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1162/2273] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:45:21 2022] DELETE /identity/v3/users/e4030d45acc64a9686ac130a4a372d36 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:45:22.559111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:22.561208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:22.575641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6b076-d6c4-458e-b024-1a770c66b328 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:22.614018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:22.637041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:22.637687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:22.638759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:22.638759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] PATH_INFO: `/v3/projects/df89794fec88414e88f7a4b8ecbce47f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:22.639554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] RBAC: Authorizing `identity:delete_project(project_id=df89794fec88414e88f7a4b8ecbce47f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:22.641467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:22.783167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df89794fec88414e88f7a4b8ecbce47f', 'request_id': 'req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.789188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df89794fec88414e88f7a4b8ecbce47f', 'request_id': 'req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.796487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df89794fec88414e88f7a4b8ecbce47f', 'request_id': 'req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.886022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] The token cache is being invalidate because project df89794fec88414e88f7a4b8ecbce47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:45:22.886350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.886970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c2bb9ccd-5da7-44a1-8dc1-5df61d45066f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:45:22.888570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1112/2274] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:45:22 2022] DELETE /identity/v3/projects/df89794fec88414e88f7a4b8ecbce47f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:45:22.893541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117494c-6e03-4b43-8279-38c37daf8e33 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:22.894011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117494c-6e03-4b43-8279-38c37daf8e33 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:22.894105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117494c-6e03-4b43-8279-38c37daf8e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:22.913483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c117494c-6e03-4b43-8279-38c37daf8e33 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:22.914719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c117494c-6e03-4b43-8279-38c37daf8e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:22.975315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1163/2275] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:22.983998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcecd10-6835-4a6f-9ac1-e385f390a687 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:22.984281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcecd10-6835-4a6f-9ac1-e385f390a687 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:22.984558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcecd10-6835-4a6f-9ac1-e385f390a687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.004706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-afcecd10-6835-4a6f-9ac1-e385f390a687 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:23.005911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-afcecd10-6835-4a6f-9ac1-e385f390a687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:23.018015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1113/2276] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:23.020284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117494c-6e03-4b43-8279-38c37daf8e33 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:23.031059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117494c-6e03-4b43-8279-38c37daf8e33 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:23.036357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:23.036907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.037184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.037457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.038178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:45:23.039138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:23.040128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:23.059824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1164/2277] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:45:23.060949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcecd10-6835-4a6f-9ac1-e385f390a687 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:23.078072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcecd10-6835-4a6f-9ac1-e385f390a687 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:23.084431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:23.085282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.085789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.087096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.087980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:23.091822 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:23.197277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1114/2278] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:45:23 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:45:23.199185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:23.211592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6864522-c3c1-4465-9196-f0f8223e4eb1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:23.253185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:23.272004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:23.272630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.272921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.273179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.273852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:23.276029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:23.392568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1165/2279] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:45:23 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:45:23.394021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:23.405066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258ab1-d0aa-4c39-925c-04fcf6522e58 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:23.409663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:23.410426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.410743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.411147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.411855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:23.413427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:23.424679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1115/2280] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:45:23.428860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:23.439047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c843852-d264-4e3f-aadb-63829344b310 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:23.456598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83daf9-0795-4593-a363-c88d9ca153c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:23.456598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83daf9-0795-4593-a363-c88d9ca153c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.456598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83daf9-0795-4593-a363-c88d9ca153c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.456598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83daf9-0795-4593-a363-c88d9ca153c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.456920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83daf9-0795-4593-a363-c88d9ca153c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:23.456920 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83daf9-0795-4593-a363-c88d9ca153c2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:23.458767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1166/2281] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:45:23.463352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:23.474401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4a38a-b9bf-4f6d-8eda-3fb3672802dc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:23.479249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:23.485245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.485245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.485245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] PATH_INFO: `/v3/projects/5d10772d974f47d7bc82c0f0b8a8facf/users/7b63aaa3a9db4fa4b50bf93eae5a1ffc/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.497398 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d10772d974f47d7bc82c0f0b8a8facf, user_id=7b63aaa3a9db4fa4b50bf93eae5a1ffc, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:23.500434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:23.590655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1116/2282] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:45:23 2022] PUT /identity/v3/projects/5d10772d974f47d7bc82c0f0b8a8facf/users/7b63aaa3a9db4fa4b50bf93eae5a1ffc/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:45:23.596572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.597100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.597276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.627185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:23.638862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:23.713922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1167/2283] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:45:23.736953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:23.738853 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:23.803045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aded58-4181-4ddd-b6d6-6dc749bb7a8b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:23.807243 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38163bc7-238f-4a6b-8865-b47cf40a81e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:23.807871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38163bc7-238f-4a6b-8865-b47cf40a81e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:23.808154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38163bc7-238f-4a6b-8865-b47cf40a81e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:23.808429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38163bc7-238f-4a6b-8865-b47cf40a81e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:23.824422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38163bc7-238f-4a6b-8865-b47cf40a81e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:23.824422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38163bc7-238f-4a6b-8865-b47cf40a81e5 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:23.830834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1117/2284] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:25.940517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:25.941978 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:26.059198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468c81-cbf4-4d6c-98f5-1e505edf385a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:26.067968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:26.069430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:26.069939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:26.070465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:26.072800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:26.088229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:26.090257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:26.130358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1168/2285] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:45:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:29.871776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782f495-025e-4a99-8a9f-d64b182bb007 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:29.872342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782f495-025e-4a99-8a9f-d64b182bb007 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:29.873061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782f495-025e-4a99-8a9f-d64b182bb007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:29.892708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:29.924750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f782f495-025e-4a99-8a9f-d64b182bb007 None None] MFA Rules not processed for user `7b63aaa3a9db4fa4b50bf93eae5a1ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:29.925968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f782f495-025e-4a99-8a9f-d64b182bb007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:30.081522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1118/2286] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:30.083619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:30.159369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:30.161863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca5f2-8d14-4104-bb92-b84ed47abe3a service nova] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:30.181724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:30.193990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:30.196081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:30.196081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:30.196081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:30.199677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:30.373445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1169/2287] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:45:30 2022] POST /identity/v3/projects => generated 371 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:45:30.374187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782f495-025e-4a99-8a9f-d64b182bb007 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:30.386467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782f495-025e-4a99-8a9f-d64b182bb007 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:30.448785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:30.449734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:30.450107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:30.451299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:30.452058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:30.454110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:30.536356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1119/2288] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:45:30 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:45:30.540368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:30.552939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6b6bc-26da-4bbd-a670-73a468f20076 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:30.563027 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:30.564262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:30.564560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:30.564835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:30.565547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:30.567224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:30.581484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1170/2289] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:45:30.584900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:30.603124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab9a95-7454-4d98-a6c5-22b92ad5f6da admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:30.603124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:30.603124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:30.603124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:30.603561 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] PATH_INFO: `/v3/projects/4a36d9652e80462a8e11c3b3e8a9c4a8/users/76eeb6a4cf0e44c299c476ad5e264a79/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:30.619441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a36d9652e80462a8e11c3b3e8a9c4a8, user_id=76eeb6a4cf0e44c299c476ad5e264a79, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:30.622977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:30.722490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1120/2290] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:45:30 2022] PUT /identity/v3/projects/4a36d9652e80462a8e11c3b3e8a9c4a8/users/76eeb6a4cf0e44c299c476ad5e264a79/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:45:30.724019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:30.736429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341a3d8-eaf9-4ae6-886b-951022436afb admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:30.811134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fe1bc-27c4-401c-8a93-15740fc371cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:30.811753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fe1bc-27c4-401c-8a93-15740fc371cd admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:30.812022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fe1bc-27c4-401c-8a93-15740fc371cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:30.812294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fe1bc-27c4-401c-8a93-15740fc371cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:30.812982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fe1bc-27c4-401c-8a93-15740fc371cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:30.815025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fe1bc-27c4-401c-8a93-15740fc371cd admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:30.826510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1171/2291] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:45:30.830550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:30.841166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dbaba-964c-464b-8a58-7f8ff9009b17 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:30.846076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:30.846777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:30.847056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:30.847331 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] PATH_INFO: `/v3/projects/4a36d9652e80462a8e11c3b3e8a9c4a8/users/76eeb6a4cf0e44c299c476ad5e264a79/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:30.849381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a36d9652e80462a8e11c3b3e8a9c4a8, user_id=76eeb6a4cf0e44c299c476ad5e264a79, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:30.852156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:30.863671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1121/2292] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:45:30 2022] PUT /identity/v3/projects/4a36d9652e80462a8e11c3b3e8a9c4a8/users/76eeb6a4cf0e44c299c476ad5e264a79/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:45:37.792229 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82843d3-a01f-4b71-9f9e-40a6436cb6c1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:37.793380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82843d3-a01f-4b71-9f9e-40a6436cb6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:37.793682 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82843d3-a01f-4b71-9f9e-40a6436cb6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:37.829843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f82843d3-a01f-4b71-9f9e-40a6436cb6c1 None None] MFA Rules not processed for user `76eeb6a4cf0e44c299c476ad5e264a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:37.831224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f82843d3-a01f-4b71-9f9e-40a6436cb6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:37.903235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1172/2293] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:37.917260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:45:37.919786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:37.945322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:38.009839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57cf7f-e7d4-4ca5-9203-169dde2c2b86 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:45:38.015865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44da61-031f-43ce-9998-ba888c7cbd86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:45:38.016566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44da61-031f-43ce-9998-ba888c7cbd86 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:38.016738 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44da61-031f-43ce-9998-ba888c7cbd86 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:38.017012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44da61-031f-43ce-9998-ba888c7cbd86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:38.072782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44da61-031f-43ce-9998-ba888c7cbd86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:45:38.074458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44da61-031f-43ce-9998-ba888c7cbd86 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:45:38.084314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1122/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:45:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:45:40.607999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:40.608591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:40.608591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:40.628965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] MFA Rules not processed for user `7b63aaa3a9db4fa4b50bf93eae5a1ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:40.630215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:40.637281 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1173/2295] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:45:48.080276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=53 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:50.622584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:45:51.735812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:45:51.736378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:45:51.736378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:45:51.756455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] MFA Rules not processed for user `c29a83c5d2ce48eda587f2cec0e959ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:45:51.757718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:45:51.815344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1123/2296] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:46:01.807714 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=25 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:46:27.215640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:27.217060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:27.295764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27971b3e-408d-48f7-85a2-07bfc82bf7d4 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:27.299913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:27.300537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:27.300816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:27.301089 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:27.310904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:27.312964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:27.323684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1174/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:46:31.825298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:31.826717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:31.908030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b5521-f8ad-489f-a47a-124510b51470 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:31.911609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:31.912219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:31.912510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:31.912809 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:31.914592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:31.973602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:31.975073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:31.984338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1124/2298] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:46:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:46:37.318896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=35 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:46:41.979961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=57 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:46:53.988316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:53.996724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a32e9-456f-4302-9038-fca8e25754fc service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:54.000384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:54.001380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:54.001380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:54.001624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:54.003862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:54.069146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:54.070693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:54.080189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1175/2299] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:46:57.617803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:57.619389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:57.690247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68821686-6fb0-41c0-95a8-42dc38773204 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:57.695081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:57.695797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:57.696068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:57.696338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:57.698031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:57.709500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:57.711153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:57.720982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1125/2300] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:46:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:46:57.738400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:57.739869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:57.752607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8878def-d6be-4db1-8564-9e8e3b283f3e service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:57.758047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:57.758740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:57.759029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:57.759304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] PATH_INFO: `/v3/users/906ca078a25447b3b9c01f4fc76e7090` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:57.760500 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] RBAC: Authorizing `identity:delete_user(user_id=906ca078a25447b3b9c01f4fc76e7090)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:57.762459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:57.945348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906ca078a25447b3b9c01f4fc76e7090'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:57.958082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906ca078a25447b3b9c01f4fc76e7090'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.028658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906ca078a25447b3b9c01f4fc76e7090'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.029196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906ca078a25447b3b9c01f4fc76e7090'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.029786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cd263623-9969-4307-9de2-006298d9869c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906ca078a25447b3b9c01f4fc76e7090'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.042123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1176/2301] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:46:57 2022] DELETE /identity/v3/users/906ca078a25447b3b9c01f4fc76e7090 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:46:58.361506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:58.371701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aece3e-fba8-4b92-a0e0-eca27d065224 service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:58.432076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:58.432711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:58.432984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:58.433257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] PATH_INFO: `/v3/projects/a0d880114866423480268022d7dd20c5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:58.434427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d880114866423480268022d7dd20c5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:58.436309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:58.559559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d880114866423480268022d7dd20c5', 'request_id': 'req-f65cd958-2049-45f4-8c16-833d1721be7d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.565412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d880114866423480268022d7dd20c5', 'request_id': 'req-f65cd958-2049-45f4-8c16-833d1721be7d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.572210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d880114866423480268022d7dd20c5', 'request_id': 'req-f65cd958-2049-45f4-8c16-833d1721be7d'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.634992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] The token cache is being invalidate because project a0d880114866423480268022d7dd20c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:46:58.635690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.636187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f65cd958-2049-45f4-8c16-833d1721be7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:46:58.637773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1126/2302] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:46:58 2022] DELETE /identity/v3/projects/a0d880114866423480268022d7dd20c5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:46:58.642936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5d5fd-a4b1-492e-9b2e-fdb68163f9fe None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:58.643751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5d5fd-a4b1-492e-9b2e-fdb68163f9fe None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:58.644031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5d5fd-a4b1-492e-9b2e-fdb68163f9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:58.709762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5ed5d5fd-a4b1-492e-9b2e-fdb68163f9fe None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:46:58.711029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5ed5d5fd-a4b1-492e-9b2e-fdb68163f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:58.742314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:46:58.772044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1177/2303] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:46:58.780000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd2d71-467a-4ace-a5d7-d6e650ec2b5b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:58.780286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd2d71-467a-4ace-a5d7-d6e650ec2b5b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:58.780563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd2d71-467a-4ace-a5d7-d6e650ec2b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:58.800358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-05fd2d71-467a-4ace-a5d7-d6e650ec2b5b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:46:58.801558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-05fd2d71-467a-4ace-a5d7-d6e650ec2b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:58.809874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1127/2304] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:46:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:46:58.814082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5d5fd-a4b1-492e-9b2e-fdb68163f9fe None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:58.824416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5d5fd-a4b1-492e-9b2e-fdb68163f9fe None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:58.828994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:58.829595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:58.829875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:58.830172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:58.831611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:46:58.831918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:58.833572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:58.845566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1178/2305] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:46:58.849773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd2d71-467a-4ace-a5d7-d6e650ec2b5b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:58.858945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd2d71-467a-4ace-a5d7-d6e650ec2b5b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:58.863677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:58.864277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:58.864555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:58.864826 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:58.865492 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:58.867432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:58.878952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:46:58.938200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1128/2306] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:46:58 2022] POST /identity/v3/projects => generated 365 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:46:58.943078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:58.953061 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836af8d6-a0c8-478b-97ab-bfdebe58767e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.008770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.009375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.009650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.009918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.010595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.012512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.102702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1179/2307] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:46:58 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:46:59.106862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.117521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba856c7-701f-4686-8a7a-474486b98d51 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.121995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.122624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.122908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.123181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.123863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.125453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.135923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1129/2308] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:46:59.140078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.150508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52f194-bb72-4363-8524-a923607e71c5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.155069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d052689-7c18-4d58-bc55-96485f9df47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.155667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d052689-7c18-4d58-bc55-96485f9df47d admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.155947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d052689-7c18-4d58-bc55-96485f9df47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.156221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d052689-7c18-4d58-bc55-96485f9df47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.156909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d052689-7c18-4d58-bc55-96485f9df47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.158609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d052689-7c18-4d58-bc55-96485f9df47d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.169151 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1180/2309] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:46:59.173129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.183916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b97be-8133-411d-8a2d-499ad2b5b72b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.188530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.189185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.189470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.190414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] PATH_INFO: `/v3/projects/424259ec24be41888d4d58d8c303f3ca/users/b3dfb365a8ef4b9597ae294c21326456/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.205913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=424259ec24be41888d4d58d8c303f3ca, user_id=b3dfb365a8ef4b9597ae294c21326456, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.208603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.294381 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1130/2310] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:46:59 2022] PUT /identity/v3/projects/424259ec24be41888d4d58d8c303f3ca/users/b3dfb365a8ef4b9597ae294c21326456/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:46:59.306088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd69ec0-0ebc-4a2b-94b8-450a0d97206e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.306411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd69ec0-0ebc-4a2b-94b8-450a0d97206e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.306707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd69ec0-0ebc-4a2b-94b8-450a0d97206e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.334955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-bcd69ec0-0ebc-4a2b-94b8-450a0d97206e None None] MFA Rules not processed for user `b3dfb365a8ef4b9597ae294c21326456`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:46:59.336219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bcd69ec0-0ebc-4a2b-94b8-450a0d97206e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:46:59.400001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1181/2311] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:46:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:46:59.404643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.465576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8460ec-981a-4733-9fce-8d3c25ec52b0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.470295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.470896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.471662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.471929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.472585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.474499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.549922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1131/2312] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:46:59 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:46:59.554371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd69ec0-0ebc-4a2b-94b8-450a0d97206e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.565152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd69ec0-0ebc-4a2b-94b8-450a0d97206e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.594515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:46:59.622748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.623371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.623840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.624165 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.625177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.627212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.706710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1182/2313] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:46:59 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:46:59.710599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.721824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbaa5e-c014-44ee-a885-587be667700b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.727928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.727928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.727928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.727928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.728423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.730001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.740627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1132/2314] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:46:59.744817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.755051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba3842-81ff-4056-8750-7f941968ca79 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.759659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.760302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.760580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.760849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] PATH_INFO: `/v3/projects/5f43f82cb40e4c13b468d5b2a31c2b8a/users/5e04f36ce72945e9a98facacf14b7841/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.776414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f43f82cb40e4c13b468d5b2a31c2b8a, user_id=5e04f36ce72945e9a98facacf14b7841, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.779113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.840256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1183/2315] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:46:59 2022] PUT /identity/v3/projects/5f43f82cb40e4c13b468d5b2a31c2b8a/users/5e04f36ce72945e9a98facacf14b7841/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:46:59.844421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.855356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfad2a-809c-4b7a-b067-d3a42d2f490b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.911329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f95c9-7004-4e95-a800-05a13715c572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.911973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f95c9-7004-4e95-a800-05a13715c572 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.912256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f95c9-7004-4e95-a800-05a13715c572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.912528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f95c9-7004-4e95-a800-05a13715c572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.913219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f95c9-7004-4e95-a800-05a13715c572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.914852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f95c9-7004-4e95-a800-05a13715c572 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.924510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1133/2316] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:46:59.928644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:46:59.938725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e98af-e5bf-4271-b666-d7d9cb791fde admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:46:59.943390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:46:59.944040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.944316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.944590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] PATH_INFO: `/v3/projects/5f43f82cb40e4c13b468d5b2a31c2b8a/users/5e04f36ce72945e9a98facacf14b7841/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.946540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f43f82cb40e4c13b468d5b2a31c2b8a, user_id=5e04f36ce72945e9a98facacf14b7841, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:46:59.949160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:46:59.959787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1184/2317] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:46:59 2022] PUT /identity/v3/projects/5f43f82cb40e4c13b468d5b2a31c2b8a/users/5e04f36ce72945e9a98facacf14b7841/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:46:59.969873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37aa3a-f6d7-4487-b467-6e647a445665 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:46:59.970189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37aa3a-f6d7-4487-b467-6e647a445665 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:46:59.970466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37aa3a-f6d7-4487-b467-6e647a445665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:46:59.998264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1e37aa3a-f6d7-4487-b467-6e647a445665 None None] MFA Rules not processed for user `5e04f36ce72945e9a98facacf14b7841`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:46:59.999452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1e37aa3a-f6d7-4487-b467-6e647a445665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:00.011577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:00.067162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1134/2318] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:46:59 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:00.078289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:00.079973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:00.138424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1777af-9b25-4091-ba37-3292a0266fb5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:00.142126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:00.142755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:00.143559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:00.143943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:00.152997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:00.154458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:00.164201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1185/2319] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:00 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:00.348903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37aa3a-f6d7-4487-b467-6e647a445665 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:00.358535 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37aa3a-f6d7-4487-b467-6e647a445665 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:00.362652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b861e69-ba74-4f0c-a069-a1bc61bf1c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:00.363250 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861e69-ba74-4f0c-a069-a1bc61bf1c21 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:00.363531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861e69-ba74-4f0c-a069-a1bc61bf1c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:00.363804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b861e69-ba74-4f0c-a069-a1bc61bf1c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:00.419235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b861e69-ba74-4f0c-a069-a1bc61bf1c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:00.420680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b861e69-ba74-4f0c-a069-a1bc61bf1c21 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:00.430245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1135/2320] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:10.160017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:10.426335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=50 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:14.446514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:14.507754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3af07-8cf2-4c9b-a313-3f4f2dd1a4ed service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:14.512161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:14.512785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:14.513064 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:14.513337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] PATH_INFO: `/v3/users/b3dfb365a8ef4b9597ae294c21326456` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:14.514515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3dfb365a8ef4b9597ae294c21326456)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:14.516442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:14.716850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3dfb365a8ef4b9597ae294c21326456'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:14.729515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3dfb365a8ef4b9597ae294c21326456'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:14.799866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3dfb365a8ef4b9597ae294c21326456'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:14.800425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3dfb365a8ef4b9597ae294c21326456'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:14.801033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3dfb365a8ef4b9597ae294c21326456'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:14.813591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1186/2321] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:14 2022] DELETE /identity/v3/users/b3dfb365a8ef4b9597ae294c21326456 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:14.818200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:14.821455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:14.821455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:14.838099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:14.839625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:14.896518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1136/2322] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:14.906578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:14.907987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:14.965528 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796281b0-da58-4cd1-8039-d425bccad718 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:14.969154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:14.969758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:14.970038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:14.970340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:14.979710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:14.981136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:14.990441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1187/2323] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:15.378491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:15.380081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:15.398611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f046f-7106-4e6a-bdd6-eb7e7833bf6f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:15.403640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:15.404422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:15.404807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:15.405188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] PATH_INFO: `/v3/projects/424259ec24be41888d4d58d8c303f3ca` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:15.406522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=424259ec24be41888d4d58d8c303f3ca)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:15.408630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:15.543067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '424259ec24be41888d4d58d8c303f3ca', 'request_id': 'req-7680b33e-ed11-482f-833d-f219aaf161b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.548671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '424259ec24be41888d4d58d8c303f3ca', 'request_id': 'req-7680b33e-ed11-482f-833d-f219aaf161b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.555546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '424259ec24be41888d4d58d8c303f3ca', 'request_id': 'req-7680b33e-ed11-482f-833d-f219aaf161b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.642496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] The token cache is being invalidate because project 424259ec24be41888d4d58d8c303f3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:15.642742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.643360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.644916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1137/2324] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:15 2022] DELETE /identity/v3/projects/424259ec24be41888d4d58d8c303f3ca => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:15.649756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:15.659263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93ba37-b865-4a02-8749-26b1c34b9a66 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:15.678103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:15.716187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:15.716822 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:15.717102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:15.717377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] PATH_INFO: `/v3/users/5e04f36ce72945e9a98facacf14b7841` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:15.718557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e04f36ce72945e9a98facacf14b7841)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:15.720484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:15.893973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e04f36ce72945e9a98facacf14b7841'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.906466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e04f36ce72945e9a98facacf14b7841'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.968919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e04f36ce72945e9a98facacf14b7841'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.969561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e04f36ce72945e9a98facacf14b7841'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.970191 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-254ecd63-1ac1-447d-bc10-1065599df53d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e04f36ce72945e9a98facacf14b7841'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:15.982758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1188/2325] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:15 2022] DELETE /identity/v3/users/5e04f36ce72945e9a98facacf14b7841 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:16.401525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:16.403015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:16.417973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680b33e-ed11-482f-833d-f219aaf161b7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:16.481991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:16.482663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:16.482955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:16.483216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] PATH_INFO: `/v3/projects/5f43f82cb40e4c13b468d5b2a31c2b8a` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:16.484367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f43f82cb40e4c13b468d5b2a31c2b8a)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:16.486344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:16.576519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:16.593193 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f43f82cb40e4c13b468d5b2a31c2b8a', 'request_id': 'req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:16.598776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f43f82cb40e4c13b468d5b2a31c2b8a', 'request_id': 'req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:16.606296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f43f82cb40e4c13b468d5b2a31c2b8a', 'request_id': 'req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:16.694629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] The token cache is being invalidate because project 5f43f82cb40e4c13b468d5b2a31c2b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:16.694863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:16.695516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-aeeb10cc-f813-4cda-8036-c2d25a2f9b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:16.697038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1138/2326] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:16 2022] DELETE /identity/v3/projects/5f43f82cb40e4c13b468d5b2a31c2b8a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:16.701952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd86380-7e2c-42bd-8d1c-438573ce0f45 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:16.702441 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd86380-7e2c-42bd-8d1c-438573ce0f45 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:16.702538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd86380-7e2c-42bd-8d1c-438573ce0f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:16.722590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-7bd86380-7e2c-42bd-8d1c-438573ce0f45 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:16.735045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7bd86380-7e2c-42bd-8d1c-438573ce0f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:16.785691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1189/2327] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:16.795810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6020b-0112-428f-9432-2eab82019dfb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:16.796099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6020b-0112-428f-9432-2eab82019dfb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:16.796374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6020b-0112-428f-9432-2eab82019dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:16.817554 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2db6020b-0112-428f-9432-2eab82019dfb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:16.819055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2db6020b-0112-428f-9432-2eab82019dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:16.826816 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1139/2328] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:16.831180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd86380-7e2c-42bd-8d1c-438573ce0f45 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:16.841463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd86380-7e2c-42bd-8d1c-438573ce0f45 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:16.846369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:16.846969 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:16.847245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:16.847514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:16.848202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:47:16.848503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:16.850750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:16.862110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1190/2329] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:47:16.867081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6020b-0112-428f-9432-2eab82019dfb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:16.876124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6020b-0112-428f-9432-2eab82019dfb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:16.880857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:16.881497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:16.881791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:16.882067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:16.882763 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:16.884681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.001555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1140/2330] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:47:16 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:17.005776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.015807 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012c096-d058-48f0-b19e-236f1cfc2b7d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.042530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:17.072601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.073180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.073463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.073736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.074435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.076389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.164104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1191/2331] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:47:17 2022] POST /identity/v3/users => generated 282 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:17.168042 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.178797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87ca61-faa4-47d3-bb1f-0dad15382868 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.183361 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.185234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.185234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.185234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.185234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.186803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.197886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1141/2332] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:17.201995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.212425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e09d6-8617-4849-9f63-4d7cf491d02c admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.217568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868e00e-ac4b-43fa-b502-2752a413b88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.218242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868e00e-ac4b-43fa-b502-2752a413b88e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.218521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868e00e-ac4b-43fa-b502-2752a413b88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.218791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868e00e-ac4b-43fa-b502-2752a413b88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.219569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868e00e-ac4b-43fa-b502-2752a413b88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.221149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868e00e-ac4b-43fa-b502-2752a413b88e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.231849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1192/2333] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:17.236158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.248355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31a37d-f9f4-46eb-b5c5-c29b3c190425 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.251699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.252349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.252631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.252895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] PATH_INFO: `/v3/projects/eb1ec40ef76f41a99be2c06103f6fcd9/users/33585f4934c144bab0669ab800b7f5af/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.268194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1ec40ef76f41a99be2c06103f6fcd9, user_id=33585f4934c144bab0669ab800b7f5af, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.270907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.378387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1142/2334] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:17 2022] PUT /identity/v3/projects/eb1ec40ef76f41a99be2c06103f6fcd9/users/33585f4934c144bab0669ab800b7f5af/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:17.393330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839bdbd-c1b4-49e8-a3b0-6a4a53872953 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.393671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839bdbd-c1b4-49e8-a3b0-6a4a53872953 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.393949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839bdbd-c1b4-49e8-a3b0-6a4a53872953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.422066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b839bdbd-c1b4-49e8-a3b0-6a4a53872953 None None] MFA Rules not processed for user `33585f4934c144bab0669ab800b7f5af`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:17.423298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b839bdbd-c1b4-49e8-a3b0-6a4a53872953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:17.490018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1193/2335] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:17.495134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.562713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f423d2-ef7c-44fe-9656-5e34aad9e215 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.572365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.573130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.573412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.573706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.574397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.576350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.664994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1143/2336] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:47:17 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:17.669083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839bdbd-c1b4-49e8-a3b0-6a4a53872953 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.680891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839bdbd-c1b4-49e8-a3b0-6a4a53872953 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.742637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.743468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.743600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.743811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.744472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.746444 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.864201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1194/2337] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:47:17 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:17.868194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.881493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b841e-63f6-457a-9a78-d0359477841d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.886464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.887133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.887595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.887974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.888742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.890468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:17.902500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1144/2338] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:47:17.906782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:17.917970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca0cd-1557-4660-8137-c9da738a3e4a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:17.922753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:17.923412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:17.923715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:17.924173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] PATH_INFO: `/v3/projects/c60f3e97d4f349269866091a4898c4ef/users/291a50ea075f48c79ffce0bf1cdaa060/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:17.940821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60f3e97d4f349269866091a4898c4ef, user_id=291a50ea075f48c79ffce0bf1cdaa060, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:17.943663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:18.017193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1195/2339] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:17 2022] PUT /identity/v3/projects/c60f3e97d4f349269866091a4898c4ef/users/291a50ea075f48c79ffce0bf1cdaa060/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:18.021113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:18.032205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944606b9-f2bf-4a33-8e17-bd8c1d1517b5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:18.052366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:18.090404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8d59a-6be6-4a96-a398-013acc92816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:18.090948 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8d59a-6be6-4a96-a398-013acc92816e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:18.091224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8d59a-6be6-4a96-a398-013acc92816e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:18.091496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8d59a-6be6-4a96-a398-013acc92816e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:18.092174 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8d59a-6be6-4a96-a398-013acc92816e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:18.093740 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8d59a-6be6-4a96-a398-013acc92816e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:18.103659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1145/2340] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:18.107935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:18.113218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:18.119098 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b2f9e-2cf3-4351-9f2e-efc820326eff admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:18.123721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:18.124358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:18.129045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:18.129045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] PATH_INFO: `/v3/projects/c60f3e97d4f349269866091a4898c4ef/users/291a50ea075f48c79ffce0bf1cdaa060/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:18.129045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60f3e97d4f349269866091a4898c4ef, user_id=291a50ea075f48c79ffce0bf1cdaa060, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:18.129574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:18.140276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1196/2341] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:18 2022] PUT /identity/v3/projects/c60f3e97d4f349269866091a4898c4ef/users/291a50ea075f48c79ffce0bf1cdaa060/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:18.151085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebef4f-ef44-4ddb-b628-3b1944354c62 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:18.151502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebef4f-ef44-4ddb-b628-3b1944354c62 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:18.151789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebef4f-ef44-4ddb-b628-3b1944354c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:18.182695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f6ebef4f-ef44-4ddb-b628-3b1944354c62 None None] MFA Rules not processed for user `291a50ea075f48c79ffce0bf1cdaa060`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:18.183935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f6ebef4f-ef44-4ddb-b628-3b1944354c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:18.252895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1146/2342] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:18.265096 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:18.266831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:18.326559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eee431-fe4e-4166-baee-078d58052e61 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:18.330244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:18.330861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:18.331146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:18.331419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:18.344210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:18.345802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:18.356822 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1197/2343] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:18.583597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebef4f-ef44-4ddb-b628-3b1944354c62 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:18.593828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebef4f-ef44-4ddb-b628-3b1944354c62 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:18.597378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72976d6a-7112-4975-81ec-e90680ff1b14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:18.597980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976d6a-7112-4975-81ec-e90680ff1b14 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:18.600950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976d6a-7112-4975-81ec-e90680ff1b14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:18.600950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72976d6a-7112-4975-81ec-e90680ff1b14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:18.655435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72976d6a-7112-4975-81ec-e90680ff1b14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:18.656871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72976d6a-7112-4975-81ec-e90680ff1b14 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:18.665750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1147/2344] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:18 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:22.212496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:22.274420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfada8a-6ed7-4072-8173-7183d718d7b2 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:22.279028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:22.279657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:22.279931 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:22.280206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] PATH_INFO: `/v3/users/33585f4934c144bab0669ab800b7f5af` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:22.281351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33585f4934c144bab0669ab800b7f5af)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:22.283354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:22.516572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33585f4934c144bab0669ab800b7f5af'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:22.530676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33585f4934c144bab0669ab800b7f5af'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:22.609066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33585f4934c144bab0669ab800b7f5af'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:22.609705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33585f4934c144bab0669ab800b7f5af'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:22.610910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33585f4934c144bab0669ab800b7f5af'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:22.630081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1198/2345] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:22 2022] DELETE /identity/v3/users/33585f4934c144bab0669ab800b7f5af => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:22.632319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:22.632800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:22.633333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:22.657669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:22.658918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:22.701720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:22.720549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1148/2346] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:22.731238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:22.732712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:22.747002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:22.791285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e24d0-9915-4c2e-a39b-633dfeb5f03e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:22.794808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:22.795419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:22.795698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:22.795970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:22.805854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:22.807327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:22.816655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1199/2347] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:23.422870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:23.424355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:23.437249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8a297-ace8-4d13-bb24-d197e313c0ae None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:23.450180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:23.450845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:23.451123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:23.451691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] PATH_INFO: `/v3/projects/eb1ec40ef76f41a99be2c06103f6fcd9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:23.452596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb1ec40ef76f41a99be2c06103f6fcd9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:23.454544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:23.644913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb1ec40ef76f41a99be2c06103f6fcd9', 'request_id': 'req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:23.651296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb1ec40ef76f41a99be2c06103f6fcd9', 'request_id': 'req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:23.658306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb1ec40ef76f41a99be2c06103f6fcd9', 'request_id': 'req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:23.750467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] The token cache is being invalidate because project eb1ec40ef76f41a99be2c06103f6fcd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:23.750467 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:23.751223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:23.756568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1149/2348] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:23 2022] DELETE /identity/v3/projects/eb1ec40ef76f41a99be2c06103f6fcd9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:23.757563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:23.767631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fff95-b186-413d-9249-242ac0c48b0c service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:23.832188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:23.832867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:23.833143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:23.833436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] PATH_INFO: `/v3/users/291a50ea075f48c79ffce0bf1cdaa060` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:23.835700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291a50ea075f48c79ffce0bf1cdaa060)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:23.837721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:24.051164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291a50ea075f48c79ffce0bf1cdaa060'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.063673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291a50ea075f48c79ffce0bf1cdaa060'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.160366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291a50ea075f48c79ffce0bf1cdaa060'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.161388 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291a50ea075f48c79ffce0bf1cdaa060'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.162481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9e36bb73-4092-40ad-9000-997a330adea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291a50ea075f48c79ffce0bf1cdaa060'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.180940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1200/2349] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:23 2022] DELETE /identity/v3/users/291a50ea075f48c79ffce0bf1cdaa060 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:24.679376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:24.680861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:24.702215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c94f3-534d-4eaf-9faf-bc5fd76940fb admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:24.765548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:24.766283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:24.766752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:24.767032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] PATH_INFO: `/v3/projects/c60f3e97d4f349269866091a4898c4ef` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:24.768199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60f3e97d4f349269866091a4898c4ef)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:24.770183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:24.880319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60f3e97d4f349269866091a4898c4ef', 'request_id': 'req-7b970734-edda-40e3-bfb9-fbf2a9bd490e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.886616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60f3e97d4f349269866091a4898c4ef', 'request_id': 'req-7b970734-edda-40e3-bfb9-fbf2a9bd490e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.893797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60f3e97d4f349269866091a4898c4ef', 'request_id': 'req-7b970734-edda-40e3-bfb9-fbf2a9bd490e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.971034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:24.976585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] The token cache is being invalidate because project c60f3e97d4f349269866091a4898c4ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:24.980026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.980026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7b970734-edda-40e3-bfb9-fbf2a9bd490e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:24.980026 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1150/2350] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:24 2022] DELETE /identity/v3/projects/c60f3e97d4f349269866091a4898c4ef => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:24.984420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3ecc2-5557-4b8d-860c-8dda82145c90 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:24.984850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3ecc2-5557-4b8d-860c-8dda82145c90 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:24.985055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3ecc2-5557-4b8d-860c-8dda82145c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.006400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a6d3ecc2-5557-4b8d-860c-8dda82145c90 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:25.007782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a6d3ecc2-5557-4b8d-860c-8dda82145c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:25.043136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:25.079492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1201/2351] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:25.088415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d127727-9709-4b19-b294-c992b41929f3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.088694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d127727-9709-4b19-b294-c992b41929f3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.088972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d127727-9709-4b19-b294-c992b41929f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.110260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3d127727-9709-4b19-b294-c992b41929f3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:25.111427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3d127727-9709-4b19-b294-c992b41929f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:25.120493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1151/2352] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:25.124879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3ecc2-5557-4b8d-860c-8dda82145c90 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:25.135076 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3ecc2-5557-4b8d-860c-8dda82145c90 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:25.139943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:25.140555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.140967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.141363 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.142225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:47:25.142610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:25.144312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:25.158008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1202/2353] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:47:25.163670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d127727-9709-4b19-b294-c992b41929f3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:25.173624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d127727-9709-4b19-b294-c992b41929f3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:25.178902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:25.179856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.179856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.180043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.180687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:25.182737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:25.279791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1152/2354] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:47:25 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:25.284138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:25.294335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b18c-3145-4652-897a-64c4d671fae3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:25.353980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:25.354635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.354895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.355163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.355825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:25.357778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:25.474775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1203/2355] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:47:25 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:25.479165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:25.489499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9be96-c4de-4428-a3f5-81deec42624a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:25.494594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:25.495212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.495526 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.495805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.496501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:25.498205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:25.508831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1153/2356] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:25.512871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:25.523175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fc791-4e4a-4d7b-ad61-f5fcbcdf5085 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:25.527867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24d0e8-6bb2-4172-9647-b58dfc9bd627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:25.528469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24d0e8-6bb2-4172-9647-b58dfc9bd627 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.528748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24d0e8-6bb2-4172-9647-b58dfc9bd627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.529019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24d0e8-6bb2-4172-9647-b58dfc9bd627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.529857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24d0e8-6bb2-4172-9647-b58dfc9bd627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:25.531433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24d0e8-6bb2-4172-9647-b58dfc9bd627 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:25.542423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1204/2357] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:25.546518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:25.558088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf13547-3c3b-4ce9-beae-242334e00ee2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:25.562921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:25.563595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.563866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.564138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] PATH_INFO: `/v3/projects/8fa45d0a944548669bafff16a95cf9ca/users/b9c29fe1f2384c169d51bc6d04978518/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.580373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa45d0a944548669bafff16a95cf9ca, user_id=b9c29fe1f2384c169d51bc6d04978518, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:25.583220 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:25.655278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1154/2358] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:25 2022] PUT /identity/v3/projects/8fa45d0a944548669bafff16a95cf9ca/users/b9c29fe1f2384c169d51bc6d04978518/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:25.666519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a609119-5234-4ec9-8397-fbbffd8c1f52 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.666806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a609119-5234-4ec9-8397-fbbffd8c1f52 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.667117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a609119-5234-4ec9-8397-fbbffd8c1f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.696818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5a609119-5234-4ec9-8397-fbbffd8c1f52 None None] MFA Rules not processed for user `b9c29fe1f2384c169d51bc6d04978518`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:25.698022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5a609119-5234-4ec9-8397-fbbffd8c1f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:25.770809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1205/2359] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:25.776254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:25.843574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a50174-5618-4aa3-8a97-4f3c4cd82d7c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:25.848434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:25.849052 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:25.849335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:25.850175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:25.850935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:25.852888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:25.989376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1155/2360] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:47:25 2022] POST /identity/v3/projects => generated 383 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:25.991745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a609119-5234-4ec9-8397-fbbffd8c1f52 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:26.004802 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a609119-5234-4ec9-8397-fbbffd8c1f52 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:26.042686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:26.074790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:26.075565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.075951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.076332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.077101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:26.079173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:26.180071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1206/2361] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:47:25 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:26.184197 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:26.196435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bfbe9-f974-4014-85de-930b7b0f523b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:26.201829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:26.203270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.203641 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.204857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.205843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:26.208033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:26.227971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1156/2362] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:47:26.229907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:26.241128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f993e-29f9-4f89-8b01-4b3cbc4ac916 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:26.247964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:26.249168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.249458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.249768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] PATH_INFO: `/v3/projects/1ca4a0b22908437eb0ceacd1221d6f3b/users/8ab0a767067a492bb05b43591f4f89a1/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.269251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca4a0b22908437eb0ceacd1221d6f3b, user_id=8ab0a767067a492bb05b43591f4f89a1, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:26.273829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:26.393056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1207/2363] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:26 2022] PUT /identity/v3/projects/1ca4a0b22908437eb0ceacd1221d6f3b/users/8ab0a767067a492bb05b43591f4f89a1/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:26.398084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:26.408130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d53e1-c7d4-4dfa-a162-1ecf4ec56351 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:26.468731 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287858d-74f0-43b3-8e0b-d606ba9f13ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:26.469349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287858d-74f0-43b3-8e0b-d606ba9f13ff admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.469630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287858d-74f0-43b3-8e0b-d606ba9f13ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.469900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287858d-74f0-43b3-8e0b-d606ba9f13ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.470696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287858d-74f0-43b3-8e0b-d606ba9f13ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:26.472286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287858d-74f0-43b3-8e0b-d606ba9f13ff admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:26.482015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1157/2364] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:26.486221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:26.496541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ae915-5594-4e94-b6b6-748a94713b79 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:26.501324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:26.501988 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.502302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.502575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] PATH_INFO: `/v3/projects/1ca4a0b22908437eb0ceacd1221d6f3b/users/8ab0a767067a492bb05b43591f4f89a1/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.504525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca4a0b22908437eb0ceacd1221d6f3b, user_id=8ab0a767067a492bb05b43591f4f89a1, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:26.507847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:26.518000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1208/2365] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:26 2022] PUT /identity/v3/projects/1ca4a0b22908437eb0ceacd1221d6f3b/users/8ab0a767067a492bb05b43591f4f89a1/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:26.588028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cca49-2fef-4227-9b46-fff6e4e595c7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.588314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cca49-2fef-4227-9b46-fff6e4e595c7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.588593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cca49-2fef-4227-9b46-fff6e4e595c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.596861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:26.616907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ab5cca49-2fef-4227-9b46-fff6e4e595c7 None None] MFA Rules not processed for user `8ab0a767067a492bb05b43591f4f89a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:26.618112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ab5cca49-2fef-4227-9b46-fff6e4e595c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:26.685462 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1158/2366] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:26.697466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:26.699411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:26.759741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c7ea-a5f0-4aa3-9415-66c12d6b47e9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:26.763322 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dcbe45-82f9-40ea-a61d-71991d1feac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:26.763927 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcbe45-82f9-40ea-a61d-71991d1feac4 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.764240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcbe45-82f9-40ea-a61d-71991d1feac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.764547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dcbe45-82f9-40ea-a61d-71991d1feac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.776269 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcbe45-82f9-40ea-a61d-71991d1feac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:26.777947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dcbe45-82f9-40ea-a61d-71991d1feac4 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:26.789013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1209/2367] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:26.901429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cca49-2fef-4227-9b46-fff6e4e595c7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:26.910883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cca49-2fef-4227-9b46-fff6e4e595c7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:26.915389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:26.916002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:26.916282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:26.916557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:26.975121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:26.976617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:26.985550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1159/2368] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:28.636956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71918993-d95e-4c98-a0b6-3981a4c199c0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:28.637420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71918993-d95e-4c98-a0b6-3981a4c199c0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:28.637534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71918993-d95e-4c98-a0b6-3981a4c199c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:28.656873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-71918993-d95e-4c98-a0b6-3981a4c199c0 None None] MFA Rules not processed for user `c370dd8fa3aa407ca22cce8425fcbc25`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:28.662429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-71918993-d95e-4c98-a0b6-3981a4c199c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:28.718402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1210/2369] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:30.094914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:30.156958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1726aa7-86ef-4181-bfe9-820051b0c5a7 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:30.161505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:30.162665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:30.162957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:30.163192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] PATH_INFO: `/v3/users/b9c29fe1f2384c169d51bc6d04978518` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:30.164430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c29fe1f2384c169d51bc6d04978518)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:30.166379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:30.377943 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:30.390997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c29fe1f2384c169d51bc6d04978518'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:30.403508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c29fe1f2384c169d51bc6d04978518'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:30.472022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c29fe1f2384c169d51bc6d04978518'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:30.480494 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c29fe1f2384c169d51bc6d04978518'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:30.481165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c29fe1f2384c169d51bc6d04978518'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:30.487177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1160/2370] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:30 2022] DELETE /identity/v3/users/b9c29fe1f2384c169d51bc6d04978518 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:30.492113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:30.492652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:30.492652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:30.511560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:30.515070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:30.539947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:30.585703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1211/2371] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:47:30.595900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:30.597448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:30.658363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996332-a51f-44c5-b35f-1029026a48d6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:30.662581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:30.663313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:30.663600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:30.663879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:30.675916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:30.675916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:30.685276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1161/2372] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:31.187919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:31.189430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:31.202552 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17a0d0-9b5a-4711-b442-bd5858fa85c4 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:31.208047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:31.208265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:31.208543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:31.209100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] PATH_INFO: `/v3/projects/8fa45d0a944548669bafff16a95cf9ca` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:31.210288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa45d0a944548669bafff16a95cf9ca)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:31.212202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:31.326150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa45d0a944548669bafff16a95cf9ca', 'request_id': 'req-03c1a9cc-3ef6-4471-87c6-0463aea18c11'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.331712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa45d0a944548669bafff16a95cf9ca', 'request_id': 'req-03c1a9cc-3ef6-4471-87c6-0463aea18c11'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.417093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa45d0a944548669bafff16a95cf9ca', 'request_id': 'req-03c1a9cc-3ef6-4471-87c6-0463aea18c11'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.429737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] The token cache is being invalidate because project 8fa45d0a944548669bafff16a95cf9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:31.430031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.430764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.432214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1212/2373] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:31 2022] DELETE /identity/v3/projects/8fa45d0a944548669bafff16a95cf9ca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:31.436460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:31.447104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a819c-6275-4edc-8d33-00e5d5839584 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:31.513762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:31.513762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:31.514117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:31.514117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] PATH_INFO: `/v3/users/8ab0a767067a492bb05b43591f4f89a1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:31.516011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab0a767067a492bb05b43591f4f89a1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:31.517958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:31.916866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab0a767067a492bb05b43591f4f89a1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.921842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:31.930166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab0a767067a492bb05b43591f4f89a1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.980532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab0a767067a492bb05b43591f4f89a1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.988188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab0a767067a492bb05b43591f4f89a1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.988846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b63f9fa5-0306-439f-ade2-4ee26e59ced4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab0a767067a492bb05b43591f4f89a1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:31.995124 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1162/2374] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:31 2022] DELETE /identity/v3/users/8ab0a767067a492bb05b43591f4f89a1 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:32.417264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:32.426315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:32.455028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1a9cc-3ef6-4471-87c6-0463aea18c11 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:32.524728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:32.526817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:32.527458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:32.527891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] PATH_INFO: `/v3/projects/1ca4a0b22908437eb0ceacd1221d6f3b` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:32.529481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca4a0b22908437eb0ceacd1221d6f3b)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:32.531465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:32.642644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca4a0b22908437eb0ceacd1221d6f3b', 'request_id': 'req-cfe011e6-d3b0-42ce-8a95-093147a1da34'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:32.647417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:32.650268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca4a0b22908437eb0ceacd1221d6f3b', 'request_id': 'req-cfe011e6-d3b0-42ce-8a95-093147a1da34'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:32.740328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca4a0b22908437eb0ceacd1221d6f3b', 'request_id': 'req-cfe011e6-d3b0-42ce-8a95-093147a1da34'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:32.766463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] The token cache is being invalidate because project 1ca4a0b22908437eb0ceacd1221d6f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:32.766903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:32.767661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfe011e6-d3b0-42ce-8a95-093147a1da34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:32.769608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1213/2375] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:32 2022] DELETE /identity/v3/projects/1ca4a0b22908437eb0ceacd1221d6f3b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:32.775223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f4ba5-e289-4803-8f1a-eb5d808c19fe None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:32.780006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f4ba5-e289-4803-8f1a-eb5d808c19fe None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:32.780006 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f4ba5-e289-4803-8f1a-eb5d808c19fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:32.813166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2d7f4ba5-e289-4803-8f1a-eb5d808c19fe None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:32.814434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2d7f4ba5-e289-4803-8f1a-eb5d808c19fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:32.901360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1163/2376] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:32.909263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217701-55fb-4060-94f2-17241c7e047f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:32.909546 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217701-55fb-4060-94f2-17241c7e047f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:32.909828 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc217701-55fb-4060-94f2-17241c7e047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:32.932273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fc217701-55fb-4060-94f2-17241c7e047f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:32.934434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fc217701-55fb-4060-94f2-17241c7e047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:32.943236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1214/2377] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:32.947729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f4ba5-e289-4803-8f1a-eb5d808c19fe None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:32.957865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f4ba5-e289-4803-8f1a-eb5d808c19fe None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:32.963297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:32.963910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:32.964218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:32.964488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:32.965451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:47:32.965755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:32.967414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:32.980587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1164/2378] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:47:32.985215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217701-55fb-4060-94f2-17241c7e047f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:32.994446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc217701-55fb-4060-94f2-17241c7e047f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:32.999509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:33.000149 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.000429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.000705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.001368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:33.003306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:33.134902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1215/2379] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:47:32 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:33.138818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:33.150887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacec38-8956-409d-aec5-c92aedd7407b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:33.218614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:33.219242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.219524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.219797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.220481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:33.222667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:33.370643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1165/2380] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:47:33 2022] POST /identity/v3/users => generated 294 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:33.374909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:33.386418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfa50f-2398-4e32-9826-2d4d8cfbe6bf admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:33.391025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:33.391632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.391909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.392179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.392872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:33.394506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:33.407147 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1216/2381] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:33.411452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:33.423749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb1ef9-e3ec-4f08-ae09-bd3f1bdff168 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:33.430385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b3525-59d9-4a0a-9e7e-cde408b511eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:33.432402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3525-59d9-4a0a-9e7e-cde408b511eb admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.432565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3525-59d9-4a0a-9e7e-cde408b511eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.432975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b3525-59d9-4a0a-9e7e-cde408b511eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.433812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3525-59d9-4a0a-9e7e-cde408b511eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:33.435891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b3525-59d9-4a0a-9e7e-cde408b511eb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:33.467076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1166/2382] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:33.471428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:33.483734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ce106-96ba-4eac-adf0-b3db3fb35241 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:33.491826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:33.495283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.495283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.495283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] PATH_INFO: `/v3/projects/293bf8daf9d24eb8ab340811d622780f/users/03f23d2dc6ae4e32b78955f0cc433651/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.513330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293bf8daf9d24eb8ab340811d622780f, user_id=03f23d2dc6ae4e32b78955f0cc433651, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:33.516953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:33.594407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1217/2383] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:33 2022] PUT /identity/v3/projects/293bf8daf9d24eb8ab340811d622780f/users/03f23d2dc6ae4e32b78955f0cc433651/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:33.605862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9ab59-7e83-4bcd-8993-1533e1319bd5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.606175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9ab59-7e83-4bcd-8993-1533e1319bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.606458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9ab59-7e83-4bcd-8993-1533e1319bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.640780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eec9ab59-7e83-4bcd-8993-1533e1319bd5 None None] MFA Rules not processed for user `03f23d2dc6ae4e32b78955f0cc433651`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:33.642039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eec9ab59-7e83-4bcd-8993-1533e1319bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:33.668152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:33.712694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1167/2384] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:33.717598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:33.782166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac251ee-2067-43f4-8591-c71a46d74bb8 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:33.787431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:33.788235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.788521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.788798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.789665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:33.791452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:33.875557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1218/2385] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:47:33 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:33.880012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9ab59-7e83-4bcd-8993-1533e1319bd5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:33.891438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9ab59-7e83-4bcd-8993-1533e1319bd5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:33.953314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:33.953936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:33.954283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:33.954543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:33.955221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:33.957152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:34.033733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1168/2386] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:47:33 2022] POST /identity/v3/users => generated 301 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:34.048335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:34.059188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0020f-f55c-4d15-bec9-e2f07b684684 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:34.086319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:34.086319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:34.086319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:34.086319 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:34.086744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:34.086744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:34.086744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1219/2387] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:34.087129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:34.098129 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27756c7-9850-4126-a3bf-de02761c03a1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:34.105577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:34.106266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:34.106547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:34.106819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] PATH_INFO: `/v3/projects/3190fba1ad834f9da9ffe6f34b59b079/users/e755f19131144b77bde4bec8a2a6203c/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:34.122758 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3190fba1ad834f9da9ffe6f34b59b079, user_id=e755f19131144b77bde4bec8a2a6203c, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:34.125428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:34.189001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1169/2388] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:34 2022] PUT /identity/v3/projects/3190fba1ad834f9da9ffe6f34b59b079/users/e755f19131144b77bde4bec8a2a6203c/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:47:34.193804 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:34.204205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b16618-d139-4bf7-86f0-c7fa1ca204f3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:34.254270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:34.264236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bf5cb-c533-4ff5-9f1f-2b50be890fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:34.264912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bf5cb-c533-4ff5-9f1f-2b50be890fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:34.265189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bf5cb-c533-4ff5-9f1f-2b50be890fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:34.265464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bf5cb-c533-4ff5-9f1f-2b50be890fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:34.266179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bf5cb-c533-4ff5-9f1f-2b50be890fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:34.267818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bf5cb-c533-4ff5-9f1f-2b50be890fa5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:34.277961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1220/2389] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:47:34.282335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:34.293709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b1fd3-f4ad-4158-9920-3f7f80795ca7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:34.298405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:34.299069 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:34.299353 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:34.299629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] PATH_INFO: `/v3/projects/3190fba1ad834f9da9ffe6f34b59b079/users/e755f19131144b77bde4bec8a2a6203c/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:34.301604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3190fba1ad834f9da9ffe6f34b59b079, user_id=e755f19131144b77bde4bec8a2a6203c, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:34.304793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:34.315255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1170/2390] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:34 2022] PUT /identity/v3/projects/3190fba1ad834f9da9ffe6f34b59b079/users/e755f19131144b77bde4bec8a2a6203c/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:34.327288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f478-0b46-49de-92c8-277d13e70f96 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:34.327605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f478-0b46-49de-92c8-277d13e70f96 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:34.327957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5f478-0b46-49de-92c8-277d13e70f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:34.356955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d4b5f478-0b46-49de-92c8-277d13e70f96 None None] MFA Rules not processed for user `e755f19131144b77bde4bec8a2a6203c`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:34.358264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d4b5f478-0b46-49de-92c8-277d13e70f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:34.430036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1221/2391] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:34.439623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:34.441546 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:34.502837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e813-79f6-4340-800e-14a76554cdd8 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:34.507368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cf6c7-dabb-4932-ae43-35e1c84193c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:34.507977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cf6c7-dabb-4932-ae43-35e1c84193c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:34.508265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cf6c7-dabb-4932-ae43-35e1c84193c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:34.508540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cf6c7-dabb-4932-ae43-35e1c84193c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:34.518343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cf6c7-dabb-4932-ae43-35e1c84193c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:34.519775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cf6c7-dabb-4932-ae43-35e1c84193c9 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:34.530545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1171/2392] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:34.666734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f478-0b46-49de-92c8-277d13e70f96 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:34.733652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5f478-0b46-49de-92c8-277d13e70f96 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:34.738275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:34.738908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:34.744958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:34.744958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] PATH_INFO: `/v3/users/03f23d2dc6ae4e32b78955f0cc433651` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:34.744958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f23d2dc6ae4e32b78955f0cc433651)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:34.744958 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:34.930749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f23d2dc6ae4e32b78955f0cc433651'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:34.942332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f23d2dc6ae4e32b78955f0cc433651'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:34.997337 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f23d2dc6ae4e32b78955f0cc433651'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:34.997954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f23d2dc6ae4e32b78955f0cc433651'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:34.998576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f23d2dc6ae4e32b78955f0cc433651'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:35.003375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:35.012556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1222/2393] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:34 2022] DELETE /identity/v3/users/03f23d2dc6ae4e32b78955f0cc433651 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:35.017093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:35.017560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:35.017671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:35.036283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:35.037594 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:35.050416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:35.097079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1172/2394] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:35.106496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:35.107912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:35.171141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b5d6c-1568-4ee9-8baf-1d752d5db446 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:35.174736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:35.175345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:35.175616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:35.175888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:35.185888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:35.187393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:35.199077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1223/2395] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:35.595991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:35.600772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:35.610031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e51eb1-9bed-4707-8801-483fd67a0908 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:35.614674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:35.615952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:35.616232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:35.616506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] PATH_INFO: `/v3/projects/293bf8daf9d24eb8ab340811d622780f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:35.617669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] RBAC: Authorizing `identity:delete_project(project_id=293bf8daf9d24eb8ab340811d622780f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:35.619697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:35.748488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293bf8daf9d24eb8ab340811d622780f', 'request_id': 'req-e268dac5-ac56-4312-b45b-658ceb1e357f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:35.753994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293bf8daf9d24eb8ab340811d622780f', 'request_id': 'req-e268dac5-ac56-4312-b45b-658ceb1e357f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:35.760417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293bf8daf9d24eb8ab340811d622780f', 'request_id': 'req-e268dac5-ac56-4312-b45b-658ceb1e357f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:35.847902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] The token cache is being invalidate because project 293bf8daf9d24eb8ab340811d622780f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:35.848194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:35.848812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:35.851167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1173/2396] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:35 2022] DELETE /identity/v3/projects/293bf8daf9d24eb8ab340811d622780f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:35.855004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:35.865498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c396d3-840e-4a70-a99f-8dc2b6a4e708 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:35.922085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:35.922746 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:35.923040 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:35.923290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] PATH_INFO: `/v3/users/e755f19131144b77bde4bec8a2a6203c` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:35.924439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e755f19131144b77bde4bec8a2a6203c)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:35.926392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:36.164908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e755f19131144b77bde4bec8a2a6203c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:36.176671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e755f19131144b77bde4bec8a2a6203c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:36.237067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e755f19131144b77bde4bec8a2a6203c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:36.237696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e755f19131144b77bde4bec8a2a6203c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:36.238340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-94e6f25c-d83b-46ad-a881-507253cc4e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e755f19131144b77bde4bec8a2a6203c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:36.252063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1224/2397] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:47:35 2022] DELETE /identity/v3/users/e755f19131144b77bde4bec8a2a6203c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:36.777110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:36.778696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:36.797445 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268dac5-ac56-4312-b45b-658ceb1e357f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:36.855906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:36.856564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:36.856840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:36.857110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] PATH_INFO: `/v3/projects/3190fba1ad834f9da9ffe6f34b59b079` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:36.858273 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3190fba1ad834f9da9ffe6f34b59b079)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:36.860159 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:36.975999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:36.978736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3190fba1ad834f9da9ffe6f34b59b079', 'request_id': 'req-12793705-bf20-4a71-8ee9-78f1d0b6431a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:36.984045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3190fba1ad834f9da9ffe6f34b59b079', 'request_id': 'req-12793705-bf20-4a71-8ee9-78f1d0b6431a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:36.990883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3190fba1ad834f9da9ffe6f34b59b079', 'request_id': 'req-12793705-bf20-4a71-8ee9-78f1d0b6431a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:37.103642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] The token cache is being invalidate because project 3190fba1ad834f9da9ffe6f34b59b079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:47:37.109105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:37.109105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-12793705-bf20-4a71-8ee9-78f1d0b6431a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:47:37.109105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1174/2398] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:47:36 2022] DELETE /identity/v3/projects/3190fba1ad834f9da9ffe6f34b59b079 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:37.111380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d532c9-14b0-46bb-b520-0a03fc05edfe None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.111794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d532c9-14b0-46bb-b520-0a03fc05edfe None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.111937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d532c9-14b0-46bb-b520-0a03fc05edfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.132877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-85d532c9-14b0-46bb-b520-0a03fc05edfe None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:37.134123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-85d532c9-14b0-46bb-b520-0a03fc05edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:37.144158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:37.196890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1225/2399] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:37.197996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ba84c-e4d7-492f-aa2c-d553d8d81c55 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.198395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ba84c-e4d7-492f-aa2c-d553d8d81c55 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.198749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ba84c-e4d7-492f-aa2c-d553d8d81c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.204826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a799eb8-ef6a-4700-ae71-dd2eb8bef379 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.205190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a799eb8-ef6a-4700-ae71-dd2eb8bef379 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.205538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a799eb8-ef6a-4700-ae71-dd2eb8bef379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.228937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-8a799eb8-ef6a-4700-ae71-dd2eb8bef379 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:37.230027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eb9ba84c-e4d7-492f-aa2c-d553d8d81c55 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:37.230174 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8a799eb8-ef6a-4700-ae71-dd2eb8bef379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:37.231275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eb9ba84c-e4d7-492f-aa2c-d553d8d81c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:37.238093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1226/2400] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:47:37.242758 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a799eb8-ef6a-4700-ae71-dd2eb8bef379 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.251658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a799eb8-ef6a-4700-ae71-dd2eb8bef379 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.256940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.257534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.257817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.258105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.259382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:47:37.259382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.260668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.273718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1227/2401] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:47:37.278001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.288776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a368-941d-4545-98e3-caf50d961720 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.293509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.294102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.294414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.294688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.295359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.297362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.300830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1175/2402] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:47:37.388736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1228/2403] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:47:37 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:47:37.390296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ba84c-e4d7-492f-aa2c-d553d8d81c55 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.401556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ba84c-e4d7-492f-aa2c-d553d8d81c55 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.457366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.457983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.458297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.458603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.459278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.461294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.474564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.476541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:37.550565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1176/2404] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:47:37 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:47:37.553600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366028-210e-44b2-b73f-795955e52a4a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.555448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.561620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d135d-ba9e-48af-a291-2b130b65e84a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.563071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d135d-ba9e-48af-a291-2b130b65e84a service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.563071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d135d-ba9e-48af-a291-2b130b65e84a service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.563071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d135d-ba9e-48af-a291-2b130b65e84a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.572415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df537ca8-9e6f-4aa9-9c2b-1eb4cfd20546 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.577580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.578356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.579000 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.579407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.580214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.581904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.594183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1177/2405] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:47:37.598312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.609158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830a262-50c6-4be7-a94d-ecf1409fa8bc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.614214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.615582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.615876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.616149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.616855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.618483 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.630054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1178/2406] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:47:37.634052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d135d-ba9e-48af-a291-2b130b65e84a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.635384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.635837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d135d-ba9e-48af-a291-2b130b65e84a service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.645255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1229/2407] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:47:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:47:37.651154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc962-c595-4118-813c-6d51cf44822a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.654242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.654909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.655776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.655776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] PATH_INFO: `/v3/projects/1e1afe245d7045de99b1c9a2d13f2822/users/ebb7349917df46f7a2b2d98a47b91c1f/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.671545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1afe245d7045de99b1c9a2d13f2822, user_id=ebb7349917df46f7a2b2d98a47b91c1f, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.675636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.752660 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1179/2408] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:47:37 2022] PUT /identity/v3/projects/1e1afe245d7045de99b1c9a2d13f2822/users/ebb7349917df46f7a2b2d98a47b91c1f/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:47:37.759115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60117d-31b6-4341-aa76-bdf806b8770e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.759561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60117d-31b6-4341-aa76-bdf806b8770e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.768238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60117d-31b6-4341-aa76-bdf806b8770e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.781072 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5f60117d-31b6-4341-aa76-bdf806b8770e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:37.782587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5f60117d-31b6-4341-aa76-bdf806b8770e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:37.796340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:37.854185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1230/2409] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:37.862793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:37.864572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:37.876517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:37.931126 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932a6c-15cc-479b-8fd6-8efadc3bcc29 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:37.935312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:37.935949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:37.936223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:37.936501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:37.948603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:37.950313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:37.961835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1180/2410] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:47:39.619538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748960f-a6ac-489c-bcd3-e9d7e4319c4f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:39.620042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748960f-a6ac-489c-bcd3-e9d7e4319c4f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:39.620126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748960f-a6ac-489c-bcd3-e9d7e4319c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:39.668267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5748960f-a6ac-489c-bcd3-e9d7e4319c4f None None] MFA Rules not processed for user `ebb7349917df46f7a2b2d98a47b91c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:39.669535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5748960f-a6ac-489c-bcd3-e9d7e4319c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:39.780231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1231/2411] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:39.794566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:39.798023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:39.878663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21264e08-db34-4183-9f56-d8d0661ecb56 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:39.882586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:39.883386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:39.883905 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:39.884168 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:39.894696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:39.896565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:39.907014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1181/2412] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:42.814104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee846b4-8255-482f-8cd0-ee88cef0c318 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:42.814756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee846b4-8255-482f-8cd0-ee88cef0c318 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:42.814859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee846b4-8255-482f-8cd0-ee88cef0c318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:42.837729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3ee846b4-8255-482f-8cd0-ee88cef0c318 None None] MFA Rules not processed for user `ebb7349917df46f7a2b2d98a47b91c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:47:42.841925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3ee846b4-8255-482f-8cd0-ee88cef0c318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:42.851109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1232/2413] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:47:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:47:45.711109 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:47:45.713556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:45.906879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd74d4-1f55-464b-b3fe-1d70ad894c0c service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:47:45.917001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc62f2a-c2d3-4b46-a888-029cad51ddad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:47:45.917856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc62f2a-c2d3-4b46-a888-029cad51ddad service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:47:45.918442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc62f2a-c2d3-4b46-a888-029cad51ddad service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:47:45.918827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc62f2a-c2d3-4b46-a888-029cad51ddad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:47:45.921084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fbc62f2a-c2d3-4b46-a888-029cad51ddad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:47:45.945311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc62f2a-c2d3-4b46-a888-029cad51ddad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:47:45.948111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc62f2a-c2d3-4b46-a888-029cad51ddad service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:47:45.966032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1182/2414] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:47:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:47:52.831070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=55 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:47:55.957636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:00.462394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80ca4a-4fea-47c2-b443-3e4302302e34 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:00.462859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80ca4a-4fea-47c2-b443-3e4302302e34 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:00.463167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80ca4a-4fea-47c2-b443-3e4302302e34 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:00.464563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1233/2415] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:48:00.470685 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34e669-6edd-4e09-bece-3bbce6ca5cc4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:00.471206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34e669-6edd-4e09-bece-3bbce6ca5cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:00.471371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34e669-6edd-4e09-bece-3bbce6ca5cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:00.504335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cb34e669-6edd-4e09-bece-3bbce6ca5cc4 None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:00.505564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cb34e669-6edd-4e09-bece-3bbce6ca5cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:00.592782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1183/2416] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:48:00.604964 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80ca4a-4fea-47c2-b443-3e4302302e34 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:00.606683 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bd80ca4a-4fea-47c2-b443-3e4302302e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:00.666625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80ca4a-4fea-47c2-b443-3e4302302e34 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:00.670241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef41e37-1977-4770-bf9f-c0e627f6b59c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:00.670859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41e37-1977-4770-bf9f-c0e627f6b59c service glance] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:00.671135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41e37-1977-4770-bf9f-c0e627f6b59c service glance] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:00.671402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef41e37-1977-4770-bf9f-c0e627f6b59c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:00.682087 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41e37-1977-4770-bf9f-c0e627f6b59c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:00.683434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef41e37-1977-4770-bf9f-c0e627f6b59c service glance] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:00.694392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1234/2417] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:48:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:48:00.958866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9befa-f063-4911-af71-f30d93f37dad None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:00.959077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9befa-f063-4911-af71-f30d93f37dad None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:00.959348 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9befa-f063-4911-af71-f30d93f37dad None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:00.960844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1184/2418] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:48:00.966563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbdf14-b052-4029-923a-e61fe9fad5b2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:00.966850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbdf14-b052-4029-923a-e61fe9fad5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:00.967183 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbdf14-b052-4029-923a-e61fe9fad5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:00.986897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-adbbdf14-b052-4029-923a-e61fe9fad5b2 None None] MFA Rules not processed for user `d5c308c33c674f1ea7111c0297174c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:00.988302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-adbbdf14-b052-4029-923a-e61fe9fad5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:00.995011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1235/2419] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:01.007458 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9befa-f063-4911-af71-f30d93f37dad None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:01.018688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9befa-f063-4911-af71-f30d93f37dad None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:01.022095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad5390-6aef-4a17-a3e1-a2d01df7c224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:01.022708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad5390-6aef-4a17-a3e1-a2d01df7c224 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:01.022984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad5390-6aef-4a17-a3e1-a2d01df7c224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:01.023252 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad5390-6aef-4a17-a3e1-a2d01df7c224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:01.030833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad5390-6aef-4a17-a3e1-a2d01df7c224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:01.032240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad5390-6aef-4a17-a3e1-a2d01df7c224 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:01.039960 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1185/2420] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:48:01.122120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb524-cdca-4bb0-8478-5285bf5e3441 None None] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:01.122431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb524-cdca-4bb0-8478-5285bf5e3441 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:01.122725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb524-cdca-4bb0-8478-5285bf5e3441 None None] PATH_INFO: `/` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:01.124109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1236/2421] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 27 08:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 27 08:48:01.129385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12bf50-ac91-4093-a38d-5feb1fe2f5fd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:01.129675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12bf50-ac91-4093-a38d-5feb1fe2f5fd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:01.129951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12bf50-ac91-4093-a38d-5feb1fe2f5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:01.151089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3d12bf50-ac91-4093-a38d-5feb1fe2f5fd None None] MFA Rules not processed for user `4e4e461b0b6142b38fd7de56395733b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:01.152325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3d12bf50-ac91-4093-a38d-5feb1fe2f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:01.161665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1186/2422] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 27 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:01.177105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb524-cdca-4bb0-8478-5285bf5e3441 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:01.188940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb524-cdca-4bb0-8478-5285bf5e3441 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:01.192469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:01.201688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:01.201688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:01.201688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:01.205997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:01.207728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:01.221492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1237/2423] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:48:06.192573 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c56be5-4fc2-44f4-af8e-d7c6315e48ff None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:06.193098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c56be5-4fc2-44f4-af8e-d7c6315e48ff None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:06.193200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c56be5-4fc2-44f4-af8e-d7c6315e48ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:06.218254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c8c56be5-4fc2-44f4-af8e-d7c6315e48ff None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:06.219473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c8c56be5-4fc2-44f4-af8e-d7c6315e48ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:06.290008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1187/2424] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:06.301763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:06.369797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe812fc2-43f1-4ac5-9e95-1392b89780fa service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:06.375341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e769c-d5f8-41b6-913c-c94b88452ac6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:06.375963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e769c-d5f8-41b6-913c-c94b88452ac6 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:06.376240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e769c-d5f8-41b6-913c-c94b88452ac6 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:06.376508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e769c-d5f8-41b6-913c-c94b88452ac6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:06.387660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e769c-d5f8-41b6-913c-c94b88452ac6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:06.390652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e769c-d5f8-41b6-913c-c94b88452ac6 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:06.400977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1238/2425] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:48:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:48:06.799074 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e10efb-a9a5-40c3-b03c-5db0dca115ea None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:06.799992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e10efb-a9a5-40c3-b03c-5db0dca115ea None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:06.800543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e10efb-a9a5-40c3-b03c-5db0dca115ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:06.847368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-41e10efb-a9a5-40c3-b03c-5db0dca115ea None None] MFA Rules not processed for user `ebb7349917df46f7a2b2d98a47b91c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:06.848637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-41e10efb-a9a5-40c3-b03c-5db0dca115ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:06.861048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1188/2426] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:16.396202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=67 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:16.820342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=77 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:28.305208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:28.305208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:28.305208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:28.320085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:28.321520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:28.330255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1239/2427] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 27 08:48:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:28.344717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e10efb-a9a5-40c3-b03c-5db0dca115ea None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:28.360645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e10efb-a9a5-40c3-b03c-5db0dca115ea None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:28.367221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:28.367940 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:28.368182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:28.368390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:28.378470 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:28.380130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:28.389970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1189/2428] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:48:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:48:38.314534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:38.386316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:39.285407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:39.286823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:39.298932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbeac1-d3b7-486d-b336-52932e5b6067 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:39.302500 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:39.303109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:39.303389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:39.303662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:39.305516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:39.317708 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:39.319299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:39.329246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1240/2429] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:48:43.223525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:43.224846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:43.239497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed002e1-5bdd-4ca3-bd1a-336dd872da91 service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:43.244003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:43.245120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:43.245399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:43.245668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] PATH_INFO: `/v3/users/c29a83c5d2ce48eda587f2cec0e959ce` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:43.246862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29a83c5d2ce48eda587f2cec0e959ce)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:43.255198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:43.434553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29a83c5d2ce48eda587f2cec0e959ce'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:43.446289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29a83c5d2ce48eda587f2cec0e959ce'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:43.524965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29a83c5d2ce48eda587f2cec0e959ce'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:43.533557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29a83c5d2ce48eda587f2cec0e959ce'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:43.534264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-241b4771-5747-4b5e-a7c4-fcddb7e0cdc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29a83c5d2ce48eda587f2cec0e959ce'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:43.540572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1190/2430] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:48:43 2022] DELETE /identity/v3/users/c29a83c5d2ce48eda587f2cec0e959ce => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:48:43.793370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:43.804845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602542ea-5b40-46c4-87e1-5069c11ce6d7 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:43.863257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:43.864015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:43.864295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:43.864583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] PATH_INFO: `/v3/projects/9dcfebb6e1774a7daed20b60d2379875` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:43.865723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dcfebb6e1774a7daed20b60d2379875)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:43.867654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:43.968488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dcfebb6e1774a7daed20b60d2379875', 'request_id': 'req-74b0ae24-256d-4dab-86f1-344bd7f0c530'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:43.974411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dcfebb6e1774a7daed20b60d2379875', 'request_id': 'req-74b0ae24-256d-4dab-86f1-344bd7f0c530'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:44.073234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dcfebb6e1774a7daed20b60d2379875', 'request_id': 'req-74b0ae24-256d-4dab-86f1-344bd7f0c530'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:44.085047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] The token cache is being invalidate because project 9dcfebb6e1774a7daed20b60d2379875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:48:44.085359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:44.086017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-74b0ae24-256d-4dab-86f1-344bd7f0c530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:48:44.087577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1241/2431] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:48:43 2022] DELETE /identity/v3/projects/9dcfebb6e1774a7daed20b60d2379875 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:48:44.093362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d519b7-a5e0-44f6-bb6c-f70189d27911 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.099915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d519b7-a5e0-44f6-bb6c-f70189d27911 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.099915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d519b7-a5e0-44f6-bb6c-f70189d27911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.114011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b6d519b7-a5e0-44f6-bb6c-f70189d27911 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:44.115267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b6d519b7-a5e0-44f6-bb6c-f70189d27911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:44.176278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1191/2432] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:44.184709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222946-86c3-4242-a3ec-b5e6e137d1a8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.184996 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222946-86c3-4242-a3ec-b5e6e137d1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.185266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f222946-86c3-4242-a3ec-b5e6e137d1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.206190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1f222946-86c3-4242-a3ec-b5e6e137d1a8 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:44.207404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1f222946-86c3-4242-a3ec-b5e6e137d1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:44.216241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1242/2433] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:44.220628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d519b7-a5e0-44f6-bb6c-f70189d27911 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:44.230257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d519b7-a5e0-44f6-bb6c-f70189d27911 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:44.235125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:44.235737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.236065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.236342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.237096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:48:44.237495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:44.239152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:44.251085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1192/2434] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:48:44.255355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f222946-86c3-4242-a3ec-b5e6e137d1a8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:44.265073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f222946-86c3-4242-a3ec-b5e6e137d1a8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:44.269742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:44.270391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.270674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.270947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.271641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:44.273549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:44.362557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1243/2435] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:48:44 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:48:44.366671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:44.376501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ced175-4398-434e-9956-ed95d3a2788b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:44.436592 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:44.437200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.437539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.437802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.438569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:44.440481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:44.448832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:44.534901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1193/2436] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:48:44 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:48:44.536088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:44.546799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f444-8f25-4ba5-800a-c1cd44535715 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:44.554737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:44.555551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.556017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.556342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.557686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:44.559334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:44.571232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1244/2437] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:48:44.576325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:44.586634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ff0bf-d21c-4c39-8dbf-ca8adfe3f326 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:44.592371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee054f8-40b5-46cd-8f81-93a08096a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:44.593008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee054f8-40b5-46cd-8f81-93a08096a8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.593286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee054f8-40b5-46cd-8f81-93a08096a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.593557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee054f8-40b5-46cd-8f81-93a08096a8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.594299 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee054f8-40b5-46cd-8f81-93a08096a8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:44.595902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee054f8-40b5-46cd-8f81-93a08096a8fb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:44.607422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1194/2438] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:48:44.611653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:44.622163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1f297-e55c-4d0e-a8f3-47759f4da380 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:44.626701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:44.627356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.627637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.627910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] PATH_INFO: `/v3/projects/b8c479ff98f24bb5807c25188580041b/users/3cb50f2c82e84eeaa6456b20f872b007/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.642801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c479ff98f24bb5807c25188580041b, user_id=3cb50f2c82e84eeaa6456b20f872b007, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:44.645470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:44.725326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1245/2439] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:48:44 2022] PUT /identity/v3/projects/b8c479ff98f24bb5807c25188580041b/users/3cb50f2c82e84eeaa6456b20f872b007/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:48:44.742354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592d5dd-571a-4943-b450-ba05c80e1a95 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.742354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592d5dd-571a-4943-b450-ba05c80e1a95 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.742354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592d5dd-571a-4943-b450-ba05c80e1a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.771581 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a592d5dd-571a-4943-b450-ba05c80e1a95 None None] MFA Rules not processed for user `3cb50f2c82e84eeaa6456b20f872b007`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:44.772823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a592d5dd-571a-4943-b450-ba05c80e1a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:44.838865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1195/2440] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:48:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:44.845697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:44.857849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:44.908566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f0cf1-bf26-4743-9a15-11de7158b4a7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:44.913325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:44.913939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:44.914283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:44.914563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:44.915271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:44.918244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:45.061398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1246/2441] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:48:44 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:48:45.062598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592d5dd-571a-4943-b450-ba05c80e1a95 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:45.073799 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592d5dd-571a-4943-b450-ba05c80e1a95 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:45.132829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:45.133450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.133727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.133997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.134696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:45.136643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:45.235622 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1196/2442] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:48:45 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:48:45.236887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:45.248255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2a061-6043-43b0-bf58-9cbe94f42619 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:45.253247 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:45.253932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.254232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.258285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.258285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:45.258285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:45.269749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1247/2443] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:48:45.274083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:45.285035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cf967-c419-46de-be26-2532925826b1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:45.289887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:45.290578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.290893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.291899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] PATH_INFO: `/v3/projects/40061b0c87b34416ac1ade5d41ba7115/users/9dfafe578e54462e9ee968e385b5b2d3/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.306966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40061b0c87b34416ac1ade5d41ba7115, user_id=9dfafe578e54462e9ee968e385b5b2d3, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:45.309668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:45.382645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1197/2444] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:48:45 2022] PUT /identity/v3/projects/40061b0c87b34416ac1ade5d41ba7115/users/9dfafe578e54462e9ee968e385b5b2d3/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:48:45.386731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:45.397010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b955fe-1557-449c-9d6a-36dbb4346882 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:45.454556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b12ace-ae1b-497e-b674-578ae1f9646a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:45.455177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b12ace-ae1b-497e-b674-578ae1f9646a admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.455455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b12ace-ae1b-497e-b674-578ae1f9646a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.455729 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b12ace-ae1b-497e-b674-578ae1f9646a admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.456419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b12ace-ae1b-497e-b674-578ae1f9646a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:45.458003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b12ace-ae1b-497e-b674-578ae1f9646a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:45.468587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1248/2445] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:48:45.472836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:45.483838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d80c-1841-4905-8e57-0bdb54033815 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:45.488566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:45.489219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.489524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.489800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] PATH_INFO: `/v3/projects/40061b0c87b34416ac1ade5d41ba7115/users/9dfafe578e54462e9ee968e385b5b2d3/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.494773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40061b0c87b34416ac1ade5d41ba7115, user_id=9dfafe578e54462e9ee968e385b5b2d3, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:45.495182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:45.505681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1198/2446] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:48:45 2022] PUT /identity/v3/projects/40061b0c87b34416ac1ade5d41ba7115/users/9dfafe578e54462e9ee968e385b5b2d3/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:48:45.516858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894dc4d-da35-468e-bf8b-2282bfacf8bc None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.517214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894dc4d-da35-468e-bf8b-2282bfacf8bc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.518436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894dc4d-da35-468e-bf8b-2282bfacf8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.557751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3894dc4d-da35-468e-bf8b-2282bfacf8bc None None] MFA Rules not processed for user `9dfafe578e54462e9ee968e385b5b2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:45.559279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3894dc4d-da35-468e-bf8b-2282bfacf8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:45.631870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1249/2447] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:48:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:45.643518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:45.645195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:45.667585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:45.701759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d27bfa-61d9-4d74-9dd6-23a751f3c692 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:45.705925 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b52feb-4d06-4165-8ebc-d82926e5ded7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:45.706559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b52feb-4d06-4165-8ebc-d82926e5ded7 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.706841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b52feb-4d06-4165-8ebc-d82926e5ded7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.707112 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b52feb-4d06-4165-8ebc-d82926e5ded7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.716400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b52feb-4d06-4165-8ebc-d82926e5ded7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:45.717886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b52feb-4d06-4165-8ebc-d82926e5ded7 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:45.727287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1199/2448] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:48:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:48:45.961651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894dc4d-da35-468e-bf8b-2282bfacf8bc None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:45.971592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894dc4d-da35-468e-bf8b-2282bfacf8bc None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:45.975180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:45.975776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:45.976051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:45.976327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:45.982100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:48:46.035283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:46.036725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:46.046329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1250/2449] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:48:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:48:51.199535 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a83d-3ada-4297-ad91-c600cf9b05f9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:51.200046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a83d-3ada-4297-ad91-c600cf9b05f9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:51.200190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71a83d-3ada-4297-ad91-c600cf9b05f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:51.219656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3b71a83d-3ada-4297-ad91-c600cf9b05f9 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:48:51.220850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3b71a83d-3ada-4297-ad91-c600cf9b05f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:51.280578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1200/2450] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:48:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:48:51.302575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:48:51.304011 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:48:51.375402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e65a4-d73b-43fb-b67f-c247fe2b46dc service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:48:51.380419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dc607-d6f6-4107-afe5-621af0fb3795 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:48:51.381036 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc607-d6f6-4107-afe5-621af0fb3795 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:48:51.381318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc607-d6f6-4107-afe5-621af0fb3795 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:48:51.381590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc607-d6f6-4107-afe5-621af0fb3795 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:48:51.390253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dc607-d6f6-4107-afe5-621af0fb3795 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:48:51.391896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dc607-d6f6-4107-afe5-621af0fb3795 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:48:51.399818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1251/2451] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:48:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:49:01.271766 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=42 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:01.396582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=56 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:02.366889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71a83d-3ada-4297-ad91-c600cf9b05f9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:02.472428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71a83d-3ada-4297-ad91-c600cf9b05f9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:02.477203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:02.477880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:02.478221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:02.478655 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] PATH_INFO: `/v3/users/3cb50f2c82e84eeaa6456b20f872b007` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:02.479840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb50f2c82e84eeaa6456b20f872b007)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:02.481967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:02.688895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb50f2c82e84eeaa6456b20f872b007'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:02.704760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb50f2c82e84eeaa6456b20f872b007'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:02.776185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb50f2c82e84eeaa6456b20f872b007'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:02.788412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb50f2c82e84eeaa6456b20f872b007'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:02.789326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb50f2c82e84eeaa6456b20f872b007'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:02.802095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1201/2452] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:02 2022] DELETE /identity/v3/users/3cb50f2c82e84eeaa6456b20f872b007 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:02.807699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:02.807699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:02.807699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:02.839773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:02.841037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:02.936447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1252/2453] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:02.952704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:02.954531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:03.025488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81b120-5c46-4198-a882-7081b1f8facf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:03.029834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:03.030510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:03.030793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:03.031066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:03.041970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:03.044269 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:03.055414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1202/2454] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:03.904530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:03.906487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:03.920750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c4169-3b48-4659-a4fe-72ea6eb691c9 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:03.926455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:03.927306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:03.927781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:03.928261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] PATH_INFO: `/v3/projects/b8c479ff98f24bb5807c25188580041b` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:03.929605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c479ff98f24bb5807c25188580041b)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:03.931598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:04.076227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c479ff98f24bb5807c25188580041b', 'request_id': 'req-42016639-2b55-4a48-8f60-437dfd721846'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.085678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c479ff98f24bb5807c25188580041b', 'request_id': 'req-42016639-2b55-4a48-8f60-437dfd721846'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.153765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c479ff98f24bb5807c25188580041b', 'request_id': 'req-42016639-2b55-4a48-8f60-437dfd721846'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.169950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] The token cache is being invalidate because project b8c479ff98f24bb5807c25188580041b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:04.170277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.173007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.174597 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1253/2455] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:03 2022] DELETE /identity/v3/projects/b8c479ff98f24bb5807c25188580041b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:04.178576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:04.191782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12faf9-231e-4d1d-83d8-33e97bcb770e service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:04.213029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:04.256025 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:04.256832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:04.257184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:04.257529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] PATH_INFO: `/v3/users/9dfafe578e54462e9ee968e385b5b2d3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:04.258791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfafe578e54462e9ee968e385b5b2d3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:04.260793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:04.495595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfafe578e54462e9ee968e385b5b2d3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.513122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfafe578e54462e9ee968e385b5b2d3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.576257 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfafe578e54462e9ee968e385b5b2d3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.586198 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfafe578e54462e9ee968e385b5b2d3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.586997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ca72acb1-7210-488a-b59a-5aef8897190e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfafe578e54462e9ee968e385b5b2d3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:04.600490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1203/2456] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:04 2022] DELETE /identity/v3/users/9dfafe578e54462e9ee968e385b5b2d3 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:05.076358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:05.081028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:05.103507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42016639-2b55-4a48-8f60-437dfd721846 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:05.204703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:05.204703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:05.204703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:05.204703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] PATH_INFO: `/v3/projects/40061b0c87b34416ac1ade5d41ba7115` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:05.206817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40061b0c87b34416ac1ade5d41ba7115)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:05.208932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:05.350422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:05.370826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40061b0c87b34416ac1ade5d41ba7115', 'request_id': 'req-998f5840-8ebf-47d2-8582-6560647ec319'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:05.377605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40061b0c87b34416ac1ade5d41ba7115', 'request_id': 'req-998f5840-8ebf-47d2-8582-6560647ec319'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:05.456359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40061b0c87b34416ac1ade5d41ba7115', 'request_id': 'req-998f5840-8ebf-47d2-8582-6560647ec319'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:05.469615 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] The token cache is being invalidate because project 40061b0c87b34416ac1ade5d41ba7115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:05.470024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:05.470933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-998f5840-8ebf-47d2-8582-6560647ec319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:05.472799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1254/2457] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:05 2022] DELETE /identity/v3/projects/40061b0c87b34416ac1ade5d41ba7115 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:05.477913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d70d8-9e4f-46fe-8178-cd30e11272ca None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:05.478422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d70d8-9e4f-46fe-8178-cd30e11272ca None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:05.478691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d70d8-9e4f-46fe-8178-cd30e11272ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:05.500617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f74d70d8-9e4f-46fe-8178-cd30e11272ca None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:05.500617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f74d70d8-9e4f-46fe-8178-cd30e11272ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:05.579613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1204/2458] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:05.592707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115a383-4801-495c-a8f8-290819e1ef03 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:05.593255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115a383-4801-495c-a8f8-290819e1ef03 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:05.593705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115a383-4801-495c-a8f8-290819e1ef03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:05.614249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2115a383-4801-495c-a8f8-290819e1ef03 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:05.615458 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2115a383-4801-495c-a8f8-290819e1ef03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:05.624025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1255/2459] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:49:05.628566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d70d8-9e4f-46fe-8178-cd30e11272ca None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:05.639717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d70d8-9e4f-46fe-8178-cd30e11272ca None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:05.647202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:05.647202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:05.647202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:05.647202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:05.648066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:49:05.648469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:05.650383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:05.666186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1205/2460] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:49:05.669976 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115a383-4801-495c-a8f8-290819e1ef03 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:05.680895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115a383-4801-495c-a8f8-290819e1ef03 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:05.686851 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:05.687480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:05.687757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:05.688025 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:05.688711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:05.690933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:05.779369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1256/2461] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:05 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:49:05.780688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:05.791476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08464f-f3d1-4564-be10-a78033355269 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:05.827630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:05.865295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:05.866076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:05.866491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:05.866954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:05.867747 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:05.869781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:05.962164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1206/2462] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:49:05 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:49:05.970427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:05.986946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d182b8-cf14-4691-8347-2578873a6825 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:05.993676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:05.994562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:05.995006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:05.995376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:05.996849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:05.998757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:06.018073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1257/2463] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:06.019929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:06.035351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87eb83-010a-41c1-8326-d200abc22661 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:06.045019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff5984-9e20-455b-ada9-9e9de239bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:06.046128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5984-9e20-455b-ada9-9e9de239bc50 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:06.046800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5984-9e20-455b-ada9-9e9de239bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:06.047205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff5984-9e20-455b-ada9-9e9de239bc50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:06.048231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff5984-9e20-455b-ada9-9e9de239bc50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:06.050589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff5984-9e20-455b-ada9-9e9de239bc50 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:06.073767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1207/2464] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:06.075503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:06.091815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e2b7b-27ec-436f-9a51-dc7a9f46188e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:06.096818 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:06.097574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:06.097929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:06.098302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] PATH_INFO: `/v3/projects/7974ec14ad6342f7858e62d3090d632d/users/0ee9441aeb88433cb974442c2c80cf97/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:06.115940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7974ec14ad6342f7858e62d3090d632d, user_id=0ee9441aeb88433cb974442c2c80cf97, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:06.118855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:06.194475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1258/2465] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:06 2022] PUT /identity/v3/projects/7974ec14ad6342f7858e62d3090d632d/users/0ee9441aeb88433cb974442c2c80cf97/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:06.205350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd7cc8-66c0-44e2-9cf3-cf65076a486d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:06.205709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd7cc8-66c0-44e2-9cf3-cf65076a486d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:06.206060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd7cc8-66c0-44e2-9cf3-cf65076a486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:06.247630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-18bd7cc8-66c0-44e2-9cf3-cf65076a486d None None] MFA Rules not processed for user `0ee9441aeb88433cb974442c2c80cf97`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:06.247630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-18bd7cc8-66c0-44e2-9cf3-cf65076a486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:06.350224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1208/2466] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:06.354809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:06.434190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bd56f-7eca-4e3b-ae87-567263840034 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:06.437760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:06.438448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:06.438730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:06.439002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:06.439694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:06.441644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:06.745665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1259/2467] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:06 2022] POST /identity/v3/projects => generated 381 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:06.748500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd7cc8-66c0-44e2-9cf3-cf65076a486d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:06.759333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd7cc8-66c0-44e2-9cf3-cf65076a486d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:06.816180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:06.816812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:06.817093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:06.817367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:06.818032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:06.819984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:06.908413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1209/2468] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:49:06 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:06.913516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:06.924631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc67aa-c6eb-407c-b0d7-dcfb54866b79 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:06.929371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:06.933806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:06.933806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:06.933806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:06.933806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:06.933806 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:06.944906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1260/2469] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:06.950923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:06.967377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117c434-a878-48b5-8547-0c4eeae8931d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:06.972155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:06.972813 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:06.973090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:06.973374 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] PATH_INFO: `/v3/projects/75f7718560784423863ac9a1b1bae469/users/104567b5885148d59f30be306b710390/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:06.991792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f7718560784423863ac9a1b1bae469, user_id=104567b5885148d59f30be306b710390, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:06.994675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:07.104647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1210/2470] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:06 2022] PUT /identity/v3/projects/75f7718560784423863ac9a1b1bae469/users/104567b5885148d59f30be306b710390/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:07.109216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:07.123106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8eb17-5667-4ebb-8556-a36d87b6e2ec admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:07.175712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:07.248299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b619434-8788-44da-9b62-47ee68591446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:07.248871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b619434-8788-44da-9b62-47ee68591446 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:07.249140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b619434-8788-44da-9b62-47ee68591446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:07.249409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b619434-8788-44da-9b62-47ee68591446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:07.250113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b619434-8788-44da-9b62-47ee68591446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:07.255078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b619434-8788-44da-9b62-47ee68591446 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:07.279455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1261/2471] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:07.280717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:07.286789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:07.292454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80ab09-169a-4aa0-8827-9377a8f2a146 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:07.298506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:07.299191 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:07.299473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:07.299759 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] PATH_INFO: `/v3/projects/75f7718560784423863ac9a1b1bae469/users/104567b5885148d59f30be306b710390/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:07.301750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f7718560784423863ac9a1b1bae469, user_id=104567b5885148d59f30be306b710390, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:07.304550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:07.317075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1211/2472] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:07 2022] PUT /identity/v3/projects/75f7718560784423863ac9a1b1bae469/users/104567b5885148d59f30be306b710390/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:07.327914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31223-ba6b-45c1-896f-3dcb29f6a072 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:07.328544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31223-ba6b-45c1-896f-3dcb29f6a072 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:07.328741 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31223-ba6b-45c1-896f-3dcb29f6a072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:07.375433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-88f31223-ba6b-45c1-896f-3dcb29f6a072 None None] MFA Rules not processed for user `104567b5885148d59f30be306b710390`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:07.375717 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-88f31223-ba6b-45c1-896f-3dcb29f6a072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:07.487252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1262/2473] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:07.507228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:07.510488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:07.588484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98884e6-bf8a-4055-bc67-85d4f90d407f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:07.594613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:07.595400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:07.595777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:07.596146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:07.606976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:07.608650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:07.619704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1212/2474] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:08.623349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f31223-ba6b-45c1-896f-3dcb29f6a072 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:08.632551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f31223-ba6b-45c1-896f-3dcb29f6a072 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:08.637190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '104567b5885148d59f30be306b710390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75f7718560784423863ac9a1b1bae469', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:08.637791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:08.638070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:08.638371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:08.639063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:49:08.639364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:08.640945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:08.652446 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1263/2475] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:49:08.657954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:08.667310 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c18cce-16e6-4145-947b-57306f649762 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:08.672441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '104567b5885148d59f30be306b710390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75f7718560784423863ac9a1b1bae469', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:08.673038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:08.673326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:08.673600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:08.674296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:08.676204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:08.787481 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1213/2476] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:08 2022] POST /identity/v3/projects => generated 353 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:08.791640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:08.802696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee582c4b-b95f-4705-a8cc-bbd89bf41067 tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:08.861748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83782a3-fbe9-4160-8685-d0e3dcf4fe0d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '104567b5885148d59f30be306b710390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75f7718560784423863ac9a1b1bae469', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:08.862428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83782a3-fbe9-4160-8685-d0e3dcf4fe0d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:08.862711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83782a3-fbe9-4160-8685-d0e3dcf4fe0d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:08.862995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83782a3-fbe9-4160-8685-d0e3dcf4fe0d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:08.863690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83782a3-fbe9-4160-8685-d0e3dcf4fe0d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:49:08.863998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83782a3-fbe9-4160-8685-d0e3dcf4fe0d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:08.865594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83782a3-fbe9-4160-8685-d0e3dcf4fe0d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:08.878843 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1264/2477] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:49:08.923888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:08.934263 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c20065-dc2e-4e85-b28f-36256a951a9d tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:08.938807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '104567b5885148d59f30be306b710390', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75f7718560784423863ac9a1b1bae469', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:08.939430 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:08.939711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:08.939982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] PATH_INFO: `/v3/projects/5986cbfe1f5e4947a33ce76f5e88ae0e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:08.941110 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5986cbfe1f5e4947a33ce76f5e88ae0e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:08.943020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:09.049972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5986cbfe1f5e4947a33ce76f5e88ae0e', 'request_id': 'req-21fcfc44-fcf9-4793-8874-201fa794aa4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:09.055629 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5986cbfe1f5e4947a33ce76f5e88ae0e', 'request_id': 'req-21fcfc44-fcf9-4793-8874-201fa794aa4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:09.062195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5986cbfe1f5e4947a33ce76f5e88ae0e', 'request_id': 'req-21fcfc44-fcf9-4793-8874-201fa794aa4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:09.191924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] The token cache is being invalidate because project 5986cbfe1f5e4947a33ce76f5e88ae0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:09.192160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:09.192791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:09.194339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1214/2478] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:08 2022] DELETE /identity/v3/projects/5986cbfe1f5e4947a33ce76f5e88ae0e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:09.292573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:09.293079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:09.293366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:09.312225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:09.313427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:09.334163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:09.368953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1265/2479] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 27 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:09.381472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:09.382973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:09.435565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:09.453487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcfc44-fcf9-4793-8874-201fa794aa4b tempest-VolumeQuotaClassesTest-2094394842 tempest-VolumeQuotaClassesTest-2094394842-project-admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:09.458784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4dcb5-2a5e-48c5-b7f3-18e2224a5dfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:09.459450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4dcb5-2a5e-48c5-b7f3-18e2224a5dfc service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:09.459893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4dcb5-2a5e-48c5-b7f3-18e2224a5dfc service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:09.460140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4dcb5-2a5e-48c5-b7f3-18e2224a5dfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:09.469784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4dcb5-2a5e-48c5-b7f3-18e2224a5dfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:09.471455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4dcb5-2a5e-48c5-b7f3-18e2224a5dfc service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:09.480841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1215/2480] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:49:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:49:09.855893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:09.857427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:09.922657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06727c5-ef84-4160-b54c-4d2a10cfdca9 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:09.927186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:09.927813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:09.928091 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:09.928387 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] PATH_INFO: `/v3/users/0ee9441aeb88433cb974442c2c80cf97` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:09.929742 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee9441aeb88433cb974442c2c80cf97)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:09.934570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:10.154101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee9441aeb88433cb974442c2c80cf97'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:10.165738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee9441aeb88433cb974442c2c80cf97'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:10.237235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee9441aeb88433cb974442c2c80cf97'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:10.237863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee9441aeb88433cb974442c2c80cf97'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:10.238509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee9441aeb88433cb974442c2c80cf97'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:10.251826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1266/2481] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:09 2022] DELETE /identity/v3/users/0ee9441aeb88433cb974442c2c80cf97 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:10.256298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:10.256814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:10.256916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:10.276217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:10.278078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:10.349876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1216/2482] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:49:10.356907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:10.358428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:10.424055 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbf1a2-9520-4aa7-b912-e839cdfd62dd admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:10.428095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:10.428774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:10.429066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:10.429344 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:10.436687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:10.443421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:10.445063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:10.456730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1267/2483] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:10.858221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:10.860256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:10.877642 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbaa3ee-6902-4c66-a6ae-e99b49ce014c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:10.883657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:10.883657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:10.883657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:10.883657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] PATH_INFO: `/v3/projects/7974ec14ad6342f7858e62d3090d632d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:10.884189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7974ec14ad6342f7858e62d3090d632d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:10.885848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:11.022098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7974ec14ad6342f7858e62d3090d632d', 'request_id': 'req-e16d6826-936f-4e54-b08f-354f92e6dc4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.027885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7974ec14ad6342f7858e62d3090d632d', 'request_id': 'req-e16d6826-936f-4e54-b08f-354f92e6dc4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.034038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7974ec14ad6342f7858e62d3090d632d', 'request_id': 'req-e16d6826-936f-4e54-b08f-354f92e6dc4b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.118748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] The token cache is being invalidate because project 7974ec14ad6342f7858e62d3090d632d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:11.118968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.119692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.122432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1217/2484] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:10 2022] DELETE /identity/v3/projects/7974ec14ad6342f7858e62d3090d632d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:11.126288 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:11.136283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520497f4-c857-4c4b-8e49-27f0e76bf501 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:11.202770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:11.203520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:11.203865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:11.204257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] PATH_INFO: `/v3/users/104567b5885148d59f30be306b710390` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:11.205484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=104567b5885148d59f30be306b710390)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:11.207430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:11.411078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104567b5885148d59f30be306b710390'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.423234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104567b5885148d59f30be306b710390'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.516847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104567b5885148d59f30be306b710390'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.517494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104567b5885148d59f30be306b710390'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.518128 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-dc134b5f-9e5e-410b-bd4a-650e02149928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104567b5885148d59f30be306b710390'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:11.532140 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1268/2485] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:11 2022] DELETE /identity/v3/users/104567b5885148d59f30be306b710390 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:12.066248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:12.066908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:12.089134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6826-936f-4e54-b08f-354f92e6dc4b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:12.159518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:12.160647 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.160931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.161203 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] PATH_INFO: `/v3/projects/75f7718560784423863ac9a1b1bae469` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.163027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f7718560784423863ac9a1b1bae469)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:12.166367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:12.176578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:12.288859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f7718560784423863ac9a1b1bae469', 'request_id': 'req-31fe1faa-31a3-410f-a470-4e0d59c30f93'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.295202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f7718560784423863ac9a1b1bae469', 'request_id': 'req-31fe1faa-31a3-410f-a470-4e0d59c30f93'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.305048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f7718560784423863ac9a1b1bae469', 'request_id': 'req-31fe1faa-31a3-410f-a470-4e0d59c30f93'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.410248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] The token cache is being invalidate because project 75f7718560784423863ac9a1b1bae469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:12.410540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.413921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-31fe1faa-31a3-410f-a470-4e0d59c30f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.413921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1218/2486] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:12 2022] DELETE /identity/v3/projects/75f7718560784423863ac9a1b1bae469 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:12.417662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a4f00-70ed-4a1c-9d71-4e607b697b42 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.418661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a4f00-70ed-4a1c-9d71-4e607b697b42 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.418766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a4f00-70ed-4a1c-9d71-4e607b697b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.441158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4f5a4f00-70ed-4a1c-9d71-4e607b697b42 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:12.442420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4f5a4f00-70ed-4a1c-9d71-4e607b697b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:12.529432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1269/2487] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:12.537289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e06ab-8e6c-4992-9c68-f17f2b3c9e36 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.537614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e06ab-8e6c-4992-9c68-f17f2b3c9e36 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.537869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e06ab-8e6c-4992-9c68-f17f2b3c9e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.560441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e80e06ab-8e6c-4992-9c68-f17f2b3c9e36 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:12.561878 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e80e06ab-8e6c-4992-9c68-f17f2b3c9e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:12.577864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1219/2488] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:12.582119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a4f00-70ed-4a1c-9d71-4e607b697b42 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:12.593947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a4f00-70ed-4a1c-9d71-4e607b697b42 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:12.599306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:12.599941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.603395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.603395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.603395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:49:12.603395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:12.603395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:12.612933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:12.616312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1270/2489] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:49:12.620700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e06ab-8e6c-4992-9c68-f17f2b3c9e36 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:12.630705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e06ab-8e6c-4992-9c68-f17f2b3c9e36 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:12.635424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:12.636031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.636312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.636686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.637483 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:12.639424 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:12.674535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:12.676299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:12.691606 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d23fc0-927a-4532-b18d-60bcdf61315e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:12.696574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:12.697209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.697492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.697764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] PATH_INFO: `/v3/users/ebb7349917df46f7a2b2d98a47b91c1f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.699816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebb7349917df46f7a2b2d98a47b91c1f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:12.701770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:12.747041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1220/2490] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:12 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:12.748294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:12.809996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a3546-3636-47da-a6fd-76a62bbe53ea admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:12.818821 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:12.820415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.820415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.820415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.821028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:12.823037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:12.898860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebb7349917df46f7a2b2d98a47b91c1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.912056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebb7349917df46f7a2b2d98a47b91c1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.922190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1221/2491] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:49:12 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:49:12.925024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:12.935459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a017a-939a-4f62-99fd-857911d0faaf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:12.940313 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:12.940928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.941206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.941482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.942219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:12.943801 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:12.957044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1222/2492] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:49:12.958471 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:12.969602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b6ac6-3b74-4775-8a87-c9b42619c6e4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:12.974417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9414cc-b264-40b9-b219-42257c7a0a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:12.975015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9414cc-b264-40b9-b219-42257c7a0a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:12.975291 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9414cc-b264-40b9-b219-42257c7a0a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:12.975589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9414cc-b264-40b9-b219-42257c7a0a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:12.976301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9414cc-b264-40b9-b219-42257c7a0a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:12.977908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9414cc-b264-40b9-b219-42257c7a0a16 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:12.990291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebb7349917df46f7a2b2d98a47b91c1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.990291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebb7349917df46f7a2b2d98a47b91c1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.990291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebb7349917df46f7a2b2d98a47b91c1f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:12.996185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1271/2493] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:12 2022] DELETE /identity/v3/users/ebb7349917df46f7a2b2d98a47b91c1f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:49:13.003329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1223/2494] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:13.007375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:13.008800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:13.076984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724df53-844d-4608-a741-b250f7d76555 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:13.081657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:13.083418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.083418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.083418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] PATH_INFO: `/v3/projects/67db1b1eacec454abe9c4dc289dd2cd6/users/0141ad290821439fa185dbee908662bf/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.098966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67db1b1eacec454abe9c4dc289dd2cd6, user_id=0141ad290821439fa185dbee908662bf, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:13.101687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:13.193187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1272/2495] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:13 2022] PUT /identity/v3/projects/67db1b1eacec454abe9c4dc289dd2cd6/users/0141ad290821439fa185dbee908662bf/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:13.198466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.199427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.199627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.218687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:13.219886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:13.250235 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:13.272488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:13.274327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:13.281335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1224/2496] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:13.289457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9fcaf-f7b4-4392-974c-be44a0ed32a3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:13.291683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:13.294216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:13.295507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:13.296155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.296432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.296702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] PATH_INFO: `/v3/projects/1e1afe245d7045de99b1c9a2d13f2822` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.297947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1afe245d7045de99b1c9a2d13f2822)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:13.300056 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:13.361761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ae4e6-443f-4f1a-8258-4337028321b3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:13.365818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e35bea-1be6-43a9-90b2-34bc5d7db770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:13.366464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e35bea-1be6-43a9-90b2-34bc5d7db770 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.366748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e35bea-1be6-43a9-90b2-34bc5d7db770 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.367008 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e35bea-1be6-43a9-90b2-34bc5d7db770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.377351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e35bea-1be6-43a9-90b2-34bc5d7db770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:13.378998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e35bea-1be6-43a9-90b2-34bc5d7db770 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:13.389407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1225/2497] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:49:13.416870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1afe245d7045de99b1c9a2d13f2822', 'request_id': 'req-f7921a54-0a38-47cc-8c53-4971f76e02c4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:13.418995 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:13.424403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1afe245d7045de99b1c9a2d13f2822', 'request_id': 'req-f7921a54-0a38-47cc-8c53-4971f76e02c4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:13.565162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1afe245d7045de99b1c9a2d13f2822', 'request_id': 'req-f7921a54-0a38-47cc-8c53-4971f76e02c4'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:13.583505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] The token cache is being invalidate because project 1e1afe245d7045de99b1c9a2d13f2822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:13.583815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:13.584499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f7921a54-0a38-47cc-8c53-4971f76e02c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:13.586833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1273/2498] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:13 2022] DELETE /identity/v3/projects/1e1afe245d7045de99b1c9a2d13f2822 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:49:13.594882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30b53-86ba-469c-9019-7f69351ec30c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.595368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30b53-86ba-469c-9019-7f69351ec30c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.595858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f30b53-86ba-469c-9019-7f69351ec30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.637632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c4f30b53-86ba-469c-9019-7f69351ec30c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:13.639523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c4f30b53-86ba-469c-9019-7f69351ec30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:13.714260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1226/2499] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:13.722895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93e769-e30b-4ae8-83a8-18103afafa4e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.723179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93e769-e30b-4ae8-83a8-18103afafa4e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.723518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93e769-e30b-4ae8-83a8-18103afafa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.745101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5f93e769-e30b-4ae8-83a8-18103afafa4e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:13.746434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5f93e769-e30b-4ae8-83a8-18103afafa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:13.754925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1274/2500] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:49:13.759152 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f30b53-86ba-469c-9019-7f69351ec30c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:13.769782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f30b53-86ba-469c-9019-7f69351ec30c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:13.774464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:13.775080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.775360 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.775628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.776318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:49:13.776619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:13.778807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:13.800958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1227/2501] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:49:13.805390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93e769-e30b-4ae8-83a8-18103afafa4e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:13.815364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93e769-e30b-4ae8-83a8-18103afafa4e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:13.820006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:13.820622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.820903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.821178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.821849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:13.824530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:13.922131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1275/2502] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:13 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:13.923351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:13.933444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb301188-d418-4052-b1d0-7e1c39054602 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:13.992814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:13.993356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:13.993636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:13.993912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:13.994605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:13.996598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:14.008092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:14.171761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1228/2503] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:49:13 2022] POST /identity/v3/users => generated 295 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:14.176029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:14.189430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e407e7-ec72-4caa-b891-9e0595c58fb2 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:14.193989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:14.194627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:14.194905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:14.198332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:14.198332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:14.198332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:14.207728 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1276/2504] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:14.212018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:14.223239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ad3d3-4f60-4969-b201-72baa46a51ef admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:14.228270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62c10b-7aff-44fb-9027-134173fc5bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:14.228884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62c10b-7aff-44fb-9027-134173fc5bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:14.229162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62c10b-7aff-44fb-9027-134173fc5bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:14.229437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62c10b-7aff-44fb-9027-134173fc5bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:14.230208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62c10b-7aff-44fb-9027-134173fc5bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:14.232787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62c10b-7aff-44fb-9027-134173fc5bb7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:14.244997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1229/2505] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:14.249347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:14.260826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35a314-a49b-4500-8a12-2d0dd31f26de admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:14.265482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:14.266308 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:14.266579 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:14.266849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] PATH_INFO: `/v3/projects/d5f098ad4f3a475580d888a59c65b13f/users/7eca31b513cd410f8fe852497dbb2018/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:14.284187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f098ad4f3a475580d888a59c65b13f, user_id=7eca31b513cd410f8fe852497dbb2018, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:14.288201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:14.408977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1277/2506] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:14 2022] PUT /identity/v3/projects/d5f098ad4f3a475580d888a59c65b13f/users/7eca31b513cd410f8fe852497dbb2018/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:14.419749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb9d4a-b370-4ff4-a14b-3249d4fa3c6d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:14.420090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb9d4a-b370-4ff4-a14b-3249d4fa3c6d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:14.420366 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb9d4a-b370-4ff4-a14b-3249d4fa3c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:14.447213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-45cb9d4a-b370-4ff4-a14b-3249d4fa3c6d None None] MFA Rules not processed for user `7eca31b513cd410f8fe852497dbb2018`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:14.448419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-45cb9d4a-b370-4ff4-a14b-3249d4fa3c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:14.511962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1230/2507] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:14.522892 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:14.524623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:14.585998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3195c65-1981-4f21-bbba-5de437b143b6 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:14.591840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:14.592527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:14.593077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:14.593584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:14.604188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:14.605575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:14.616178 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1278/2508] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:14.887840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5d640-78d9-49cf-9ded-6835d2b4846b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:14.888180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5d640-78d9-49cf-9ded-6835d2b4846b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:14.888875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5d640-78d9-49cf-9ded-6835d2b4846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:14.935958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bdd5d640-78d9-49cf-9ded-6835d2b4846b None None] MFA Rules not processed for user `0141ad290821439fa185dbee908662bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:14.937189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bdd5d640-78d9-49cf-9ded-6835d2b4846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:15.007305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1231/2509] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:15.011737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:15.014392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:15.084262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31fdfb-bdcf-46b6-9ef1-b7cc84c9f821 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:15.088854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:15.090075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:15.090486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:15.090753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:15.091986 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:15.093627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:15.184553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1279/2510] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:15 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:15.185632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5d640-78d9-49cf-9ded-6835d2b4846b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:15.205603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5d640-78d9-49cf-9ded-6835d2b4846b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:15.289039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:15.289743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:15.289941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:15.290238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:15.291189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:15.293792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:15.304843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:15.376334 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1232/2511] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:49:15 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:49:15.377825 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:15.384443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:15.391266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7eae6-b9ba-465f-ae84-53411959df5a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:15.400205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:15.400205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:15.400205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:15.400205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:15.400907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:15.401263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:15.419531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1280/2512] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:15.420788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:15.432624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5700e1-1ab5-4b0c-9762-bc189639773c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:15.439737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:15.440480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:15.440863 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:15.441241 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] PATH_INFO: `/v3/projects/6934da39543c4ef786f2dd970d6ccc28/users/ffb74b2756d447baad5ea7af2c8ea8e6/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:15.458016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6934da39543c4ef786f2dd970d6ccc28, user_id=ffb74b2756d447baad5ea7af2c8ea8e6, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:15.460817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:15.569004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1233/2513] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:15 2022] PUT /identity/v3/projects/6934da39543c4ef786f2dd970d6ccc28/users/ffb74b2756d447baad5ea7af2c8ea8e6/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:15.573102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:15.583941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0856347-f7b8-47dd-8ffe-0ce63ab57cc5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:15.640628 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11dfcc-33d8-4f92-9598-26f2aadcce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:15.641329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11dfcc-33d8-4f92-9598-26f2aadcce6a admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:15.641607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11dfcc-33d8-4f92-9598-26f2aadcce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:15.641877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11dfcc-33d8-4f92-9598-26f2aadcce6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:15.642591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11dfcc-33d8-4f92-9598-26f2aadcce6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:15.644176 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11dfcc-33d8-4f92-9598-26f2aadcce6a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:15.654022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1281/2514] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:15.660588 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:15.671003 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b7805-e97a-4a58-9048-f0329080dd0c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:15.675778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:15.676438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:15.676712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:15.676976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] PATH_INFO: `/v3/projects/6934da39543c4ef786f2dd970d6ccc28/users/ffb74b2756d447baad5ea7af2c8ea8e6/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:15.679020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6934da39543c4ef786f2dd970d6ccc28, user_id=ffb74b2756d447baad5ea7af2c8ea8e6, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:15.681626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:15.692599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1234/2515] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:15 2022] PUT /identity/v3/projects/6934da39543c4ef786f2dd970d6ccc28/users/ffb74b2756d447baad5ea7af2c8ea8e6/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:16.887715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:16.888358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:16.888463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:16.918440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] MFA Rules not processed for user `ffb74b2756d447baad5ea7af2c8ea8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:16.919854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:17.006620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1282/2516] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:17.014528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:17.077985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7844ca-3173-4064-b764-4bcac74ab7d9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:17.082578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:17.083201 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:17.083479 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:17.083753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:17.149914 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:17.151315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:17.162577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1235/2517] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:24.239821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:24.241219 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:24.252817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf60c-7f29-4c87-9997-e3297edd7f5a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:24.256235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:24.256837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:24.257114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:24.257384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:24.267113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:24.268564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:24.277873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1283/2518] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:27.158525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=3,keystone:SELECT=65 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:28.247961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:28.249709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:28.306636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3131c9e-93b4-4ac1-b39e-8f259963c85c service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:28.310343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:28.310947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:28.311761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:28.312196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:28.314119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:28.387254 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:28.388618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:28.399812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1236/2519] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:49:33.377189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:33.378735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:33.391048 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de84cad-3fb1-4700-b1ab-fc976adb8a2b service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:33.394620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625310c-be68-4d61-bc3c-a16b5de9c824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:33.395226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625310c-be68-4d61-bc3c-a16b5de9c824 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:33.395511 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625310c-be68-4d61-bc3c-a16b5de9c824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:33.395787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625310c-be68-4d61-bc3c-a16b5de9c824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:33.397474 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d625310c-be68-4d61-bc3c-a16b5de9c824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:33.410932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625310c-be68-4d61-bc3c-a16b5de9c824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:33.412358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625310c-be68-4d61-bc3c-a16b5de9c824 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:33.424201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1284/2520] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:38.394879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=54 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:42.251944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:42.315172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859d885-ece6-4cee-9b93-eb812e2d4957 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:42.319733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:42.320396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:42.320681 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:42.320959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] PATH_INFO: `/v3/users/7eca31b513cd410f8fe852497dbb2018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:42.322116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eca31b513cd410f8fe852497dbb2018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:42.324089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:42.513476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eca31b513cd410f8fe852497dbb2018'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:42.526528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eca31b513cd410f8fe852497dbb2018'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:42.615242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eca31b513cd410f8fe852497dbb2018'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:42.622963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eca31b513cd410f8fe852497dbb2018'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:42.623597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eca31b513cd410f8fe852497dbb2018'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:42.630264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1237/2521] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:42 2022] DELETE /identity/v3/users/7eca31b513cd410f8fe852497dbb2018 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:42.634814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:42.635291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:42.635409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:42.655293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:42.656619 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:42.698051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:42.715639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1285/2522] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:42.725785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:42.727218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:42.783970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f2a80-e28d-476f-b1f0-982636501f5c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:42.787531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabb4d5-3d5a-440f-95a8-718c31a0f7de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:42.788138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabb4d5-3d5a-440f-95a8-718c31a0f7de service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:42.788416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabb4d5-3d5a-440f-95a8-718c31a0f7de service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:42.788686 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabb4d5-3d5a-440f-95a8-718c31a0f7de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:42.797703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabb4d5-3d5a-440f-95a8-718c31a0f7de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:42.799156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabb4d5-3d5a-440f-95a8-718c31a0f7de service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:42.808558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1238/2523] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:43.279068 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:43.281299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:43.294924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae63f4-2677-485f-9f49-88954c93af6a None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:43.303362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:43.304024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:43.304378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:43.304561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] PATH_INFO: `/v3/projects/d5f098ad4f3a475580d888a59c65b13f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:43.306107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f098ad4f3a475580d888a59c65b13f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:43.309776 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:43.437705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f098ad4f3a475580d888a59c65b13f', 'request_id': 'req-9b9662b8-3aab-46e1-b6b2-d9bb07403046'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:43.443352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f098ad4f3a475580d888a59c65b13f', 'request_id': 'req-9b9662b8-3aab-46e1-b6b2-d9bb07403046'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:43.516160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f098ad4f3a475580d888a59c65b13f', 'request_id': 'req-9b9662b8-3aab-46e1-b6b2-d9bb07403046'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:43.527233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] The token cache is being invalidate because project d5f098ad4f3a475580d888a59c65b13f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:43.527457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:43.528085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-9b9662b8-3aab-46e1-b6b2-d9bb07403046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:43.529575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1286/2524] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:43 2022] DELETE /identity/v3/projects/d5f098ad4f3a475580d888a59c65b13f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:43.534967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add43633-d969-4599-8a6f-2daba9b6395e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:43.535436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add43633-d969-4599-8a6f-2daba9b6395e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:43.536231 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add43633-d969-4599-8a6f-2daba9b6395e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:43.555605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-add43633-d969-4599-8a6f-2daba9b6395e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:43.556875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-add43633-d969-4599-8a6f-2daba9b6395e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:43.574844 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:43.620593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1239/2525] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:43.628636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d369d-ede7-4c01-8d4c-133f757fe21d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:43.628922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d369d-ede7-4c01-8d4c-133f757fe21d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:43.629226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d369d-ede7-4c01-8d4c-133f757fe21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:43.649722 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c49d369d-ede7-4c01-8d4c-133f757fe21d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:43.650957 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c49d369d-ede7-4c01-8d4c-133f757fe21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:43.660058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1287/2526] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:43.664399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add43633-d969-4599-8a6f-2daba9b6395e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:43.675107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add43633-d969-4599-8a6f-2daba9b6395e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:43.679719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:43.680344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:43.680625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:43.680902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:43.681601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:49:43.681909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:43.683572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:43.695369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1240/2527] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:49:43.699665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d369d-ede7-4c01-8d4c-133f757fe21d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:43.709591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d369d-ede7-4c01-8d4c-133f757fe21d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:43.714248 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:43.714854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:43.715137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:43.715410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:43.716078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:43.718943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:43.803712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1288/2528] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:43 2022] POST /identity/v3/projects => generated 397 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:43.807516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:43.818004 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed64fb0-4cb9-4818-a73b-8f36b9afdc57 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:43.874342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:43.874950 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:43.875228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:43.875500 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:43.876171 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:43.878079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:44.025524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1241/2529] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:49:43 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:44.038440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:44.049375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ac069-1c26-490e-b423-be5ce99b99b4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:44.053989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:44.054562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:44.054847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:44.055126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:44.055824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:44.057447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:44.068573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1289/2530] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:44.072675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:44.083743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea3aa5-24a9-4b06-ab26-429a23fa7b98 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:44.088531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1465d72-80e4-4817-9430-b8094e5f72a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:44.089138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1465d72-80e4-4817-9430-b8094e5f72a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:44.089417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1465d72-80e4-4817-9430-b8094e5f72a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:44.089692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1465d72-80e4-4817-9430-b8094e5f72a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:44.090442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1465d72-80e4-4817-9430-b8094e5f72a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:44.092303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1465d72-80e4-4817-9430-b8094e5f72a8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:44.104804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1242/2531] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:44.109440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:44.119817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18e6c7-c3d6-43bb-a71e-a4a625c66782 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:44.125296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:44.125952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:44.126267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:44.126544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] PATH_INFO: `/v3/projects/0fd05d24d1ec4b54b4c7c9e79e665d17/users/d266f33fc6c14e148363f75f7ab38587/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:44.145906 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd05d24d1ec4b54b4c7c9e79e665d17, user_id=d266f33fc6c14e148363f75f7ab38587, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:44.148718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:44.226854 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1290/2532] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:44 2022] PUT /identity/v3/projects/0fd05d24d1ec4b54b4c7c9e79e665d17/users/d266f33fc6c14e148363f75f7ab38587/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:44.237287 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4ff2-352e-4d00-bf26-6dc385dc171a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:44.237579 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4ff2-352e-4d00-bf26-6dc385dc171a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:44.237855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4ff2-352e-4d00-bf26-6dc385dc171a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:44.266643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-140d4ff2-352e-4d00-bf26-6dc385dc171a None None] MFA Rules not processed for user `d266f33fc6c14e148363f75f7ab38587`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:44.269216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-140d4ff2-352e-4d00-bf26-6dc385dc171a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:44.346598 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1243/2533] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:44.357591 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:44.359481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:44.417501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c50c02-6c7a-4e90-9702-8350a93090e6 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:44.421643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:44.422321 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:44.422608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:44.422864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:44.431821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:44.433305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:44.443242 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1291/2534] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:47.631014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4ff2-352e-4d00-bf26-6dc385dc171a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:47.640613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4ff2-352e-4d00-bf26-6dc385dc171a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:47.644400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:47.645007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:47.645286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:47.645557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:47.647317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:47.661182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:47.727513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:47.729054 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:47.747885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1244/2535] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:54.439488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:55.063101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:55.064801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:55.138000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d1d9e-6db3-41ba-98f3-53033e8674c2 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:55.142868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:55.143505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:55.143849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:55.144052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] PATH_INFO: `/v3/users/0141ad290821439fa185dbee908662bf` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:55.145405 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0141ad290821439fa185dbee908662bf)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:55.147361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:55.232524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:55.234606 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:55.321058 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0141ad290821439fa185dbee908662bf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:55.323345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e93aa6-02a1-46e2-8892-b2eb0e924637 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:55.329785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:55.330511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:55.330955 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:55.331377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:55.333446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:55.336903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0141ad290821439fa185dbee908662bf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:55.415184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0141ad290821439fa185dbee908662bf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:55.415970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0141ad290821439fa185dbee908662bf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:55.416797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0141ad290821439fa185dbee908662bf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:55.421726 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:55.423393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:55.438054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1292/2536] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:55 2022] DELETE /identity/v3/users/0141ad290821439fa185dbee908662bf => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:55.440290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:55.449552 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1245/2537] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:49:55.506489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:55.509514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:55.600852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ffd80-ad61-483d-8cfd-789fd45dc37e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:55.609698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:55.610390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:55.610668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:55.611086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] PATH_INFO: `/v3/users/d266f33fc6c14e148363f75f7ab38587` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:55.612860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d266f33fc6c14e148363f75f7ab38587)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:55.615216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:55.629705 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:55.631847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:55.647709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:55.659748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80d205-ee07-4529-bdd4-9037db1e9a0c service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:55.738734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:55.739501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:55.739886 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:55.740253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:55.743041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:55.758498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:55.760176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:55.775806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1246/2538] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:55.858730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:55.861349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:55.888328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f1d69-99ec-474a-ad29-b18302d2bb2a service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:55.895482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:55.896411 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:55.896983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:55.897264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] PATH_INFO: `/v3/projects/67db1b1eacec454abe9c4dc289dd2cd6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:55.908837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] RBAC: Authorizing `identity:delete_project(project_id=67db1b1eacec454abe9c4dc289dd2cd6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:55.910856 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:55.916298 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd266f33fc6c14e148363f75f7ab38587'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:55.942928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd266f33fc6c14e148363f75f7ab38587'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.043888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd266f33fc6c14e148363f75f7ab38587'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.045469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd266f33fc6c14e148363f75f7ab38587'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.045469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8e84ed8b-33da-4900-aa8e-102fa96ad7ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd266f33fc6c14e148363f75f7ab38587'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.072536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1293/2539] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:55 2022] DELETE /identity/v3/users/d266f33fc6c14e148363f75f7ab38587 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:56.077563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:56.078327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:56.078966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:56.087397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67db1b1eacec454abe9c4dc289dd2cd6', 'request_id': 'req-ce68d51c-fcdc-46d4-a676-523ecfed252c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.096578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67db1b1eacec454abe9c4dc289dd2cd6', 'request_id': 'req-ce68d51c-fcdc-46d4-a676-523ecfed252c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.106070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67db1b1eacec454abe9c4dc289dd2cd6', 'request_id': 'req-ce68d51c-fcdc-46d4-a676-523ecfed252c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.119634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:56.121310 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:56.204037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1294/2540] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:56.205010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:56.206536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:56.216741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] The token cache is being invalidate because project 67db1b1eacec454abe9c4dc289dd2cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:56.218498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.219477 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.224094 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1247/2541] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:55 2022] DELETE /identity/v3/projects/67db1b1eacec454abe9c4dc289dd2cd6 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:49:56.225206 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:56.226785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:56.269385 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745a33e-9554-46bd-86d9-2f6dc3b33f1d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:56.273469 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:56.274208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:56.274497 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:56.274773 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:56.276657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:56.291744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:56.295399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:56.296744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68d51c-fcdc-46d4-a676-523ecfed252c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:56.300674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:56.307369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:56.307369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:56.307369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:56.307369 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:56.308745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1295/2542] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:56.309669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:56.311859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:56.318321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:56.319939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:56.331720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1248/2543] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:56.334097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b84af8-a0af-4ab4-81f7-ef223b42ba2b service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:56.339420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:56.340369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:56.340651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:56.341115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] PATH_INFO: `/v3/users/ffb74b2756d447baad5ea7af2c8ea8e6` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:56.343285 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb74b2756d447baad5ea7af2c8ea8e6)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:56.345544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:56.726438 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:56.726438 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb74b2756d447baad5ea7af2c8ea8e6'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.729173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb74b2756d447baad5ea7af2c8ea8e6'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.857646 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb74b2756d447baad5ea7af2c8ea8e6'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.859144 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb74b2756d447baad5ea7af2c8ea8e6'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.859913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb74b2756d447baad5ea7af2c8ea8e6'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:56.874538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1296/2544] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:56 2022] DELETE /identity/v3/users/ffb74b2756d447baad5ea7af2c8ea8e6 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:57.232473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:57.247220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:57.249151 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7043a5-a8cd-495d-9bb1-21a925190653 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:57.249316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:57.260405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:57.338999 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638caab-4a5f-4bc6-822a-c8ef13938be5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:57.345667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:57.346678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:57.346975 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:57.347312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] PATH_INFO: `/v3/projects/0fd05d24d1ec4b54b4c7c9e79e665d17` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:57.348419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:57.355302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd05d24d1ec4b54b4c7c9e79e665d17)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:57.355302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:57.355544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:57.355544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:57.355544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] PATH_INFO: `/v3/projects/6934da39543c4ef786f2dd970d6ccc28` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:57.356661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6934da39543c4ef786f2dd970d6ccc28)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:57.359145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:57.495826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd05d24d1ec4b54b4c7c9e79e665d17', 'request_id': 'req-aa254cb2-8c98-49a1-ad94-07aef5e2decc'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.503853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd05d24d1ec4b54b4c7c9e79e665d17', 'request_id': 'req-aa254cb2-8c98-49a1-ad94-07aef5e2decc'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.509286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6934da39543c4ef786f2dd970d6ccc28', 'request_id': 'req-9dbb7332-e8e0-47fd-8620-a14def0456c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.515948 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6934da39543c4ef786f2dd970d6ccc28', 'request_id': 'req-9dbb7332-e8e0-47fd-8620-a14def0456c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.523436 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6934da39543c4ef786f2dd970d6ccc28', 'request_id': 'req-9dbb7332-e8e0-47fd-8620-a14def0456c9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.580745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd05d24d1ec4b54b4c7c9e79e665d17', 'request_id': 'req-aa254cb2-8c98-49a1-ad94-07aef5e2decc'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.592810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] The token cache is being invalidate because project 0fd05d24d1ec4b54b4c7c9e79e665d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:57.593198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.593829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-aa254cb2-8c98-49a1-ad94-07aef5e2decc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.595346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1297/2545] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:57 2022] DELETE /identity/v3/projects/0fd05d24d1ec4b54b4c7c9e79e665d17 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:57.692742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] The token cache is being invalidate because project 6934da39543c4ef786f2dd970d6ccc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:49:57.692742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.692742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9dbb7332-e8e0-47fd-8620-a14def0456c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:57.692742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1249/2546] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:57 2022] DELETE /identity/v3/projects/6934da39543c4ef786f2dd970d6ccc28 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:57.693386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f96b-c0ec-45fa-afdc-a9b676a5dbd6 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:57.693386 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f96b-c0ec-45fa-afdc-a9b676a5dbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:57.697502 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f96b-c0ec-45fa-afdc-a9b676a5dbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:57.730477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5ab3f96b-c0ec-45fa-afdc-a9b676a5dbd6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:57.734772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5ab3f96b-c0ec-45fa-afdc-a9b676a5dbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:57.819135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:57.838074 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1298/2547] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:57.848757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1560f6-c664-481d-9734-3d2fd5ffe978 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:57.849039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1560f6-c664-481d-9734-3d2fd5ffe978 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:57.855237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1560f6-c664-481d-9734-3d2fd5ffe978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:57.879729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4e1560f6-c664-481d-9734-3d2fd5ffe978 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:57.883595 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4e1560f6-c664-481d-9734-3d2fd5ffe978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:57.891346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1250/2548] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:57.895754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3f96b-c0ec-45fa-afdc-a9b676a5dbd6 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:57.910251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3f96b-c0ec-45fa-afdc-a9b676a5dbd6 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:57.914083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:57.914782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:57.915059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:57.915326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:57.916017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:49:57.916320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:57.917979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:57.930069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1299/2549] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:49:57.934482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1560f6-c664-481d-9734-3d2fd5ffe978 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:57.944035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1560f6-c664-481d-9734-3d2fd5ffe978 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:57.948952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:57.949637 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:57.949985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:57.950351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:57.951019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:57.953671 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:58.053555 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1251/2550] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:49:57 2022] POST /identity/v3/projects => generated 359 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:49:58.053933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:58.063357 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458184a-2e4f-47bf-b749-7ca94ebdf8be admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:58.121746 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:58.122306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:58.122587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:58.122863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:58.123562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:58.125505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:58.242431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1300/2551] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:49:58 2022] POST /identity/v3/users => generated 280 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:49:58.247505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:58.257781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f432b-a826-470e-9827-0d2c24ef8a2e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:58.263894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:58.265544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:58.266038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:58.266562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:58.267611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:58.269422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:58.284636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1252/2552] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:58.285584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:58.298625 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f778d4-8331-4035-bc8c-c760ac7a40d7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:58.304029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ed7ad-ad3f-48d0-95fe-bf11172312e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:58.304896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed7ad-ad3f-48d0-95fe-bf11172312e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:58.305339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed7ad-ad3f-48d0-95fe-bf11172312e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:58.305771 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed7ad-ad3f-48d0-95fe-bf11172312e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:58.306665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ed7ad-ad3f-48d0-95fe-bf11172312e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:58.308489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ed7ad-ad3f-48d0-95fe-bf11172312e6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:58.321220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1301/2553] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:49:58.325409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:58.338066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd659ee-3e27-4a8a-97a7-dbd2043792da admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:58.344132 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:58.344805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:58.345085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:58.345358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] PATH_INFO: `/v3/projects/ef936bfd27cb4a39b0cf257699b2fd44/users/c1ebe2f67ec740fbbe65f3ea18897f74/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:58.362204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef936bfd27cb4a39b0cf257699b2fd44, user_id=c1ebe2f67ec740fbbe65f3ea18897f74, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:58.365077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:58.528428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1253/2554] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:49:58 2022] PUT /identity/v3/projects/ef936bfd27cb4a39b0cf257699b2fd44/users/c1ebe2f67ec740fbbe65f3ea18897f74/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:58.542162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb44ec-95ab-4c81-80f6-36a2bbb58ce4 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:58.542453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb44ec-95ab-4c81-80f6-36a2bbb58ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:58.542734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb44ec-95ab-4c81-80f6-36a2bbb58ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:58.585712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-30cb44ec-95ab-4c81-80f6-36a2bbb58ce4 None None] MFA Rules not processed for user `c1ebe2f67ec740fbbe65f3ea18897f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:58.587029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-30cb44ec-95ab-4c81-80f6-36a2bbb58ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:58.679732 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1302/2555] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:58 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:58.717634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:58.719659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:58.758239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:58.783520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d969e-29ca-47af-8302-68cea9fa0d5f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:58.788371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64822f-c888-40f2-a9b0-e7353e40fd9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:58.788999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64822f-c888-40f2-a9b0-e7353e40fd9e service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:58.789278 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64822f-c888-40f2-a9b0-e7353e40fd9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:58.790030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64822f-c888-40f2-a9b0-e7353e40fd9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:58.801048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64822f-c888-40f2-a9b0-e7353e40fd9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:58.802574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64822f-c888-40f2-a9b0-e7353e40fd9e service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:58.815786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1254/2556] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:58 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:58.834514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb44ec-95ab-4c81-80f6-36a2bbb58ce4 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:58.880899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:49:58.900933 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb44ec-95ab-4c81-80f6-36a2bbb58ce4 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:58.905981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:58.906660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:58.906943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:58.907221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] PATH_INFO: `/v3/users/c1ebe2f67ec740fbbe65f3ea18897f74` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:58.908431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ebe2f67ec740fbbe65f3ea18897f74)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:58.910492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:59.105489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ebe2f67ec740fbbe65f3ea18897f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:59.122781 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ebe2f67ec740fbbe65f3ea18897f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:59.192824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ebe2f67ec740fbbe65f3ea18897f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:59.193513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ebe2f67ec740fbbe65f3ea18897f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:59.194135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ebe2f67ec740fbbe65f3ea18897f74'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:49:59.213479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1303/2557] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:49:58 2022] DELETE /identity/v3/users/c1ebe2f67ec740fbbe65f3ea18897f74 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:49:59.218388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:59.219016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:59.219301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:59.241663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:49:59.243034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:59.304549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1255/2558] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:49:59.315223 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:59.316648 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:59.378368 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7140-ec38-4f9c-b10c-518802ae0a9f admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:59.381940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa34ce-b365-4b0b-8a14-688f619ce41b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:59.382592 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa34ce-b365-4b0b-8a14-688f619ce41b service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:59.382871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa34ce-b365-4b0b-8a14-688f619ce41b service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:59.383137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa34ce-b365-4b0b-8a14-688f619ce41b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:59.392407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa34ce-b365-4b0b-8a14-688f619ce41b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:59.393832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa34ce-b365-4b0b-8a14-688f619ce41b service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:49:59.403777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1304/2559] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:49:59.869613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:49:59.869613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:49:59.878560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30a050-ad64-4262-be76-c64017767b5a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:49:59.885708 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:49:59.886391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:49:59.887037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:49:59.887037 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] PATH_INFO: `/v3/projects/ef936bfd27cb4a39b0cf257699b2fd44` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:49:59.888099 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef936bfd27cb4a39b0cf257699b2fd44)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:49:59.890635 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.025095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef936bfd27cb4a39b0cf257699b2fd44', 'request_id': 'req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:00.031725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef936bfd27cb4a39b0cf257699b2fd44', 'request_id': 'req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:00.040372 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef936bfd27cb4a39b0cf257699b2fd44', 'request_id': 'req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:00.125836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] The token cache is being invalidate because project ef936bfd27cb4a39b0cf257699b2fd44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:50:00.125836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:00.125836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f1b0e38f-be40-45f8-9275-a7d6eb0550ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:00.128855 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1256/2560] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:49:59 2022] DELETE /identity/v3/projects/ef936bfd27cb4a39b0cf257699b2fd44 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:00.134842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d8fe3-ddc8-49ae-b2af-79fc74957678 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.135345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d8fe3-ddc8-49ae-b2af-79fc74957678 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.135429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d8fe3-ddc8-49ae-b2af-79fc74957678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.156101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-554d8fe3-ddc8-49ae-b2af-79fc74957678 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:00.157246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-554d8fe3-ddc8-49ae-b2af-79fc74957678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:00.225448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1305/2561] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:00.233288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcea179-e9d3-41d7-b595-9690cde21b20 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.233684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcea179-e9d3-41d7-b595-9690cde21b20 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.234234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcea179-e9d3-41d7-b595-9690cde21b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.254440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4fcea179-e9d3-41d7-b595-9690cde21b20 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:00.256093 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4fcea179-e9d3-41d7-b595-9690cde21b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:00.268911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1257/2562] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:00.270081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d8fe3-ddc8-49ae-b2af-79fc74957678 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.277519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:00.283786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d8fe3-ddc8-49ae-b2af-79fc74957678 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:00.289611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:00.289611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.289611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.290073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.290529 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:50:00.290834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:00.292455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.306181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1306/2563] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:50:00.310343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcea179-e9d3-41d7-b595-9690cde21b20 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.320521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcea179-e9d3-41d7-b595-9690cde21b20 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:00.325195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:00.326038 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.326991 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.327266 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.328104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:00.330022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.428513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1258/2564] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:00 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:00.433078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.443719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed89240-6851-4117-bcb2-1a2173995c1e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:00.512316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:00.513415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.514613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.515209 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.516104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:00.518501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.613562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1307/2565] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:00 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:00.614508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.626158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629466c-d8f5-488b-9a47-b38739b355b0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:00.632449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:00.633149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.633464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.633743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.634536 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:00.636249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.650249 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1259/2566] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:00.654977 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.664073 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b6733-4ee1-4df5-bd96-32c798e2f40c admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:00.668512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:00.669102 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.669388 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.669666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.670539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:00.672353 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.687497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.693724 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1308/2567] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:00.693850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:00.696295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.706890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ffa99-e6ad-4bf3-8ffe-b04b84b78c0c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:00.707922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901223-a172-4128-aaa5-b7a762fe6b2c admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:00.712223 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:00.712720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aefafc-28fb-4f9d-982a-d51e9711d5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:00.712889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.713140 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.713380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefafc-28fb-4f9d-982a-d51e9711d5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.713487 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] PATH_INFO: `/v3/users/c370dd8fa3aa407ca22cce8425fcbc25` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.713654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefafc-28fb-4f9d-982a-d51e9711d5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.713951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aefafc-28fb-4f9d-982a-d51e9711d5e9 admin admin] PATH_INFO: `/v3/projects/f8490f272c824b4bbcee1de049fc735d/users/f1d2e4c98aa140fabd34443c01074e48/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.714730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c370dd8fa3aa407ca22cce8425fcbc25)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:00.717744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.733554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aefafc-28fb-4f9d-982a-d51e9711d5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8490f272c824b4bbcee1de049fc735d, user_id=f1d2e4c98aa140fabd34443c01074e48, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:00.736273 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aefafc-28fb-4f9d-982a-d51e9711d5e9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:00.745346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:00.827052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1309/2568] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:00 2022] PUT /identity/v3/projects/f8490f272c824b4bbcee1de049fc735d/users/f1d2e4c98aa140fabd34443c01074e48/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:50:00.843827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:00.843827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:00.843827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:00.869188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] MFA Rules not processed for user `f1d2e4c98aa140fabd34443c01074e48`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:00.870451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:00.926248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c370dd8fa3aa407ca22cce8425fcbc25'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:00.936432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1310/2569] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:00.938498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c370dd8fa3aa407ca22cce8425fcbc25'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:00.947524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:00.948941 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:00.984182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:01.005349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf61a18-31f3-4e5c-a8d1-7f6e1eb6f3b5 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:01.009038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:01.009667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:01.009945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:01.010256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:01.019075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:01.020506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:01.029399 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1311/2570] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:50:01.066434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c370dd8fa3aa407ca22cce8425fcbc25'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.074021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c370dd8fa3aa407ca22cce8425fcbc25'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.074819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50b4660b-c61a-48c5-a11a-b4d47e32830f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c370dd8fa3aa407ca22cce8425fcbc25'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.081350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1260/2571] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:50:00 2022] DELETE /identity/v3/users/c370dd8fa3aa407ca22cce8425fcbc25 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:01.342990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:01.405965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901f3f9-acf2-4cda-9dea-09f7208d194d service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:01.465192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:01.465738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:01.466012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:01.466313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] PATH_INFO: `/v3/projects/940250de2f6f47ed9edc1bb0468551c8` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:01.467700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] RBAC: Authorizing `identity:delete_project(project_id=940250de2f6f47ed9edc1bb0468551c8)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:01.470136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:01.598942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940250de2f6f47ed9edc1bb0468551c8', 'request_id': 'req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.604507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940250de2f6f47ed9edc1bb0468551c8', 'request_id': 'req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.695193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940250de2f6f47ed9edc1bb0468551c8', 'request_id': 'req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.721950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] The token cache is being invalidate because project 940250de2f6f47ed9edc1bb0468551c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:50:01.721950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.721950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-21d4bd25-9b8b-43fa-a654-7d82a6ce281e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:01.721950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1312/2572] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:50:01 2022] DELETE /identity/v3/projects/940250de2f6f47ed9edc1bb0468551c8 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:01.723737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf46886-3fd2-4030-b241-7f2492445f2e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:01.724021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf46886-3fd2-4030-b241-7f2492445f2e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:01.724293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf46886-3fd2-4030-b241-7f2492445f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:01.744476 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eaf46886-3fd2-4030-b241-7f2492445f2e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:01.745702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eaf46886-3fd2-4030-b241-7f2492445f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:01.805297 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1261/2573] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:01.813343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05661f27-cb0a-47bd-b1e2-14352581e45d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:01.813627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05661f27-cb0a-47bd-b1e2-14352581e45d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:01.813899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05661f27-cb0a-47bd-b1e2-14352581e45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:01.834612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-05661f27-cb0a-47bd-b1e2-14352581e45d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:01.835956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-05661f27-cb0a-47bd-b1e2-14352581e45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:01.852205 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1313/2574] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:01.855046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf46886-3fd2-4030-b241-7f2492445f2e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:01.865019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf46886-3fd2-4030-b241-7f2492445f2e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:01.869623 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:01.870268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:01.870682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:01.870952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:01.871665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:50:01.871967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:01.873649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:01.889468 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1262/2575] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:50:01.891270 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05661f27-cb0a-47bd-b1e2-14352581e45d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:01.900972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05661f27-cb0a-47bd-b1e2-14352581e45d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:01.907129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:01.909738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:01.909738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:01.909738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:01.909738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:01.910823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:01.915329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:01.995673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1314/2576] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:01 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:01.998002 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.008614 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6e801-f7d2-46a3-9733-4541b6c639f0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.064158 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.064793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.065072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.065346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.066009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.067956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.184021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1263/2577] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:01 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:02.185316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.196561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c67f6-b032-455c-8686-ad4e5e6b632d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.201230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.201924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.202185 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.203167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.203814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.205402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.215666 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1315/2578] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:02.220969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.231262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272f963-cea3-48ac-9df7-fed5be9e0774 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.236015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.236621 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.236897 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.237173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.237892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.239510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.241891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:02.251871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1264/2579] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:02.256012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.268509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be48dcb-31e0-4618-953e-9bde7ec9dbd9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.272616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1444a0b-8c34-4d23-8150-1d4303976d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.273305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1444a0b-8c34-4d23-8150-1d4303976d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.273584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1444a0b-8c34-4d23-8150-1d4303976d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.273723 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.273857 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1444a0b-8c34-4d23-8150-1d4303976d4e admin admin] PATH_INFO: `/v3/projects/50808a876ea24416b631789b73465f7f/users/06129f511d624d8a97816e84c0b83c59/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.276472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:02.290678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1444a0b-8c34-4d23-8150-1d4303976d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50808a876ea24416b631789b73465f7f, user_id=06129f511d624d8a97816e84c0b83c59, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.293416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1444a0b-8c34-4d23-8150-1d4303976d4e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.335213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daa8f0-b0d6-4b0b-9c58-c6dcf91d7442 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.338825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.339474 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.339750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.340021 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.341775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:02.352938 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1316/2580] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:02 2022] PUT /identity/v3/projects/50808a876ea24416b631789b73465f7f/users/06129f511d624d8a97816e84c0b83c59/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:02.355285 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.356778 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.364057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac318-2dee-475d-8340-a5a299989562 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.364342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac318-2dee-475d-8340-a5a299989562 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.364613 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac318-2dee-475d-8340-a5a299989562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.368010 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1265/2581] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:02.393769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-65fac318-2dee-475d-8340-a5a299989562 None None] MFA Rules not processed for user `06129f511d624d8a97816e84c0b83c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:02.395031 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-65fac318-2dee-475d-8340-a5a299989562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:02.462239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1317/2582] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:02.467157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.478077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3652604-01d6-4943-a61c-1a74e316fa11 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.539578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.540101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.540380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.540652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.541336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.543289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.625765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1266/2583] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:02 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:02.627035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fac318-2dee-475d-8340-a5a299989562 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.638090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fac318-2dee-475d-8340-a5a299989562 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.698665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.699612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.699891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.700168 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.700833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.702863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.772486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1318/2584] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:02 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:02.773649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.785532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b5450-f155-48b9-b870-61fe1ca55c82 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.792683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.793247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.793893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.793893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.794566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.796904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.808861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1267/2585] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:50:02.813342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.831236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e90aad-f559-4050-a739-49d912eebaca admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:02.838398 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:02.839070 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:02.839349 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:02.839687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] PATH_INFO: `/v3/projects/52c5bed6e26047959a967c87d3fcb9ce/users/4af6a0aa99a1419581c69e70c0046cf0/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:02.862417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c5bed6e26047959a967c87d3fcb9ce, user_id=4af6a0aa99a1419581c69e70c0046cf0, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:02.865315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:02.964373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1319/2586] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:02 2022] PUT /identity/v3/projects/52c5bed6e26047959a967c87d3fcb9ce/users/4af6a0aa99a1419581c69e70c0046cf0/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:02.968850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:02.979236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f0369-89ed-4df6-b258-e0dd10747978 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:03.039401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaefa4-00a7-4986-9933-7c11c1d01f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:03.040142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaefa4-00a7-4986-9933-7c11c1d01f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:03.040638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaefa4-00a7-4986-9933-7c11c1d01f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:03.040974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaefa4-00a7-4986-9933-7c11c1d01f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:03.041730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaefa4-00a7-4986-9933-7c11c1d01f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:03.043498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaefa4-00a7-4986-9933-7c11c1d01f9e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:03.053734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1268/2587] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:50:03.057904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:03.065797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:03.069756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097f86d-4fde-4a0d-8103-f3bd43423891 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:03.074672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:03.075420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:03.075783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:03.076170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] PATH_INFO: `/v3/projects/52c5bed6e26047959a967c87d3fcb9ce/users/4af6a0aa99a1419581c69e70c0046cf0/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:03.078820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c5bed6e26047959a967c87d3fcb9ce, user_id=4af6a0aa99a1419581c69e70c0046cf0, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:03.081508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:03.092327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1320/2588] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:03 2022] PUT /identity/v3/projects/52c5bed6e26047959a967c87d3fcb9ce/users/4af6a0aa99a1419581c69e70c0046cf0/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:50:03.104210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:03.104503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:03.104784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:03.128496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:03.136326 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] MFA Rules not processed for user `4af6a0aa99a1419581c69e70c0046cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:03.137683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:03.224828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1269/2589] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:03.232347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:03.234756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:03.293264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966be5f-ba96-41f9-bfaf-f99227e3428b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:03.297836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:03.298414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:03.298698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:03.299166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:03.310557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:03.312782 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:03.322882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1321/2590] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:03.588096 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:03.589616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:03.611264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a480d-594d-4215-9314-ccac67dcf10b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:03.620696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:03.620870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:03.621693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:03.621693 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:03.691178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:03.692646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:03.703125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1270/2591] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:06.238726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:06.240329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:06.305111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150dfd1-4d58-4629-8ebb-828e49f5cf76 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:06.309562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:06.313480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:06.313480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:06.313480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] PATH_INFO: `/v3/users/26128a80933f4be3bed504d7a2d2785f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:06.313480 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26128a80933f4be3bed504d7a2d2785f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:06.314521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:06.535789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26128a80933f4be3bed504d7a2d2785f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:06.547883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26128a80933f4be3bed504d7a2d2785f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:06.624848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26128a80933f4be3bed504d7a2d2785f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:06.625468 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26128a80933f4be3bed504d7a2d2785f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:06.626071 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d7913241-de5c-44bc-9533-d30a4c24515a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26128a80933f4be3bed504d7a2d2785f'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:06.638917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1322/2592] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:50:06 2022] DELETE /identity/v3/users/26128a80933f4be3bed504d7a2d2785f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:06.872139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:06.885137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02f4bc-ed5c-43a7-bd12-de7c0e3e84e9 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:06.943734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:06.944373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:06.944650 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:06.944922 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] PATH_INFO: `/v3/projects/e2c60f49a9e14417bd1d9504128fc94b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:06.946062 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2c60f49a9e14417bd1d9504128fc94b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:06.947980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:07.050966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:07.059153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2c60f49a9e14417bd1d9504128fc94b', 'request_id': 'req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:07.064765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2c60f49a9e14417bd1d9504128fc94b', 'request_id': 'req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:07.070980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2c60f49a9e14417bd1d9504128fc94b', 'request_id': 'req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:07.191244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] The token cache is being invalidate because project e2c60f49a9e14417bd1d9504128fc94b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:50:07.194951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:07.194951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-02a1ddbe-5de2-4279-9bf4-a765e1fd33a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:07.194951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1271/2593] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:50:06 2022] DELETE /identity/v3/projects/e2c60f49a9e14417bd1d9504128fc94b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:07.198767 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd8f64-e725-4fbf-b10e-872c1965ee46 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.199186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd8f64-e725-4fbf-b10e-872c1965ee46 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.199466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd8f64-e725-4fbf-b10e-872c1965ee46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.222023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a2fd8f64-e725-4fbf-b10e-872c1965ee46 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:07.226389 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a2fd8f64-e725-4fbf-b10e-872c1965ee46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:07.231765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:07.286236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1323/2594] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:07.294441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182ace2-ebfa-4f7a-8a18-5b578e0b70c4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.294724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182ace2-ebfa-4f7a-8a18-5b578e0b70c4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.294995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182ace2-ebfa-4f7a-8a18-5b578e0b70c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.317378 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f182ace2-ebfa-4f7a-8a18-5b578e0b70c4 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:07.318619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f182ace2-ebfa-4f7a-8a18-5b578e0b70c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:07.326859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1272/2595] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:07.331238 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd8f64-e725-4fbf-b10e-872c1965ee46 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:07.343826 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd8f64-e725-4fbf-b10e-872c1965ee46 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:07.348810 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:07.349454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.349735 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.350020 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.350745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:50:07.351052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:07.352665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:07.365649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1324/2596] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:50:07.370648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182ace2-ebfa-4f7a-8a18-5b578e0b70c4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:07.386393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182ace2-ebfa-4f7a-8a18-5b578e0b70c4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:07.386393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:07.386393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.386393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.386903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.386903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:07.388452 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:07.487572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1273/2597] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:07 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:07.492093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:07.503233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a12f-ca49-44dd-8588-6857df8f3189 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:07.571099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:07.572445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.572445 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.572644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.573340 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:07.575318 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:07.657107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1325/2598] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:07 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:07.661081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:07.672113 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa1823-3c69-46c1-bc3d-8190394a925f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:07.676814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:07.677478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.677871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.678170 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.678866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:07.680422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:07.691842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1274/2599] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:07.699574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:07.710663 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e325b1-f12b-468a-9a19-0ac2b219b52d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:07.715430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf08be9-b14e-4d96-8a62-31e3ad70e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:07.716060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf08be9-b14e-4d96-8a62-31e3ad70e956 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.716320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf08be9-b14e-4d96-8a62-31e3ad70e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.716617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf08be9-b14e-4d96-8a62-31e3ad70e956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.717576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf08be9-b14e-4d96-8a62-31e3ad70e956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:07.720302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf08be9-b14e-4d96-8a62-31e3ad70e956 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:07.737603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1326/2600] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:07.741710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:07.753533 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2244bb-2ec5-436f-8ea1-9326d805c2cd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:07.758339 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:07.759060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.759337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.759613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] PATH_INFO: `/v3/projects/e696cc6a3e7e4d939f874cc7c14df3f0/users/d50ca5da8075414fbd2e7aee04001320/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.776188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e696cc6a3e7e4d939f874cc7c14df3f0, user_id=d50ca5da8075414fbd2e7aee04001320, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:07.778985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:07.860309 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1275/2601] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:07 2022] PUT /identity/v3/projects/e696cc6a3e7e4d939f874cc7c14df3f0/users/d50ca5da8075414fbd2e7aee04001320/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:07.871587 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:07.871876 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:07.872158 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:07.902000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] MFA Rules not processed for user `d50ca5da8075414fbd2e7aee04001320`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:07.903330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:07.971127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1327/2602] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:07.981836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:07.984242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:08.046397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1578ab-bbfc-447b-91f6-cc34fe883165 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:08.050444 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e467b6-57c4-4d79-913a-c15bf119c35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:08.051108 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e467b6-57c4-4d79-913a-c15bf119c35c service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:08.051514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e467b6-57c4-4d79-913a-c15bf119c35c service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:08.051785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e467b6-57c4-4d79-913a-c15bf119c35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:08.060228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e467b6-57c4-4d79-913a-c15bf119c35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:08.061910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e467b6-57c4-4d79-913a-c15bf119c35c service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:08.071145 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1276/2603] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:17.519244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:17.520756 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:17.531400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:17.589778 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fc0ce-4de2-473c-93eb-72c5def6295d None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:17.595115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:17.595754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:17.596030 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:17.596345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] PATH_INFO: `/v3/users/f1d2e4c98aa140fabd34443c01074e48` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:17.597498 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1d2e4c98aa140fabd34443c01074e48)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:17.599491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:17.821473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1d2e4c98aa140fabd34443c01074e48'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:17.834519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1d2e4c98aa140fabd34443c01074e48'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:17.923702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1d2e4c98aa140fabd34443c01074e48'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:17.924421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1d2e4c98aa140fabd34443c01074e48'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:17.925092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1d2e4c98aa140fabd34443c01074e48'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:17.938137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1328/2604] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:50:17 2022] DELETE /identity/v3/users/f1d2e4c98aa140fabd34443c01074e48 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:50:17.948136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:17.948620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:17.948807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:17.968674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:17.970645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:18.030646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1277/2605] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:18.040868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:18.042289 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:18.101569 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ed356-9d0a-4c4c-be37-20ec911e18ea admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:18.105314 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3568e-9974-4be7-ac16-4abf11fdef9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:18.105922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3568e-9974-4be7-ac16-4abf11fdef9b service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:18.106233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3568e-9974-4be7-ac16-4abf11fdef9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:18.106507 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3568e-9974-4be7-ac16-4abf11fdef9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:18.115922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3568e-9974-4be7-ac16-4abf11fdef9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:18.117354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3568e-9974-4be7-ac16-4abf11fdef9b service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:18.128527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1329/2606] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:18.639694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:18.639694 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:18.646968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:18.674580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46abf3-07df-4b1c-b611-0a6df94712ba None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:18.674580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:18.674580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:18.674580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:18.675425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] PATH_INFO: `/v3/projects/f8490f272c824b4bbcee1de049fc735d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:18.675425 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8490f272c824b4bbcee1de049fc735d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:18.675845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:18.790473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8490f272c824b4bbcee1de049fc735d', 'request_id': 'req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:18.795975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8490f272c824b4bbcee1de049fc735d', 'request_id': 'req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:18.805262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8490f272c824b4bbcee1de049fc735d', 'request_id': 'req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:18.892911 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] The token cache is being invalidate because project f8490f272c824b4bbcee1de049fc735d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:50:18.893230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:18.898015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f52f8cbe-c9fe-4d86-b8a3-75e65edc09e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:18.898015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1278/2607] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:50:18 2022] DELETE /identity/v3/projects/f8490f272c824b4bbcee1de049fc735d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:18.901884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dac6b7-643d-471d-9f45-943481f6c8a6 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:18.902403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dac6b7-643d-471d-9f45-943481f6c8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:18.902517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dac6b7-643d-471d-9f45-943481f6c8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:18.922812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e4dac6b7-643d-471d-9f45-943481f6c8a6 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:18.924047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e4dac6b7-643d-471d-9f45-943481f6c8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:18.949555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:18.982339 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1330/2608] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:19.048881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4c542-e503-40bd-b73c-5a9f37430834 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.049409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4c542-e503-40bd-b73c-5a9f37430834 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.049409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4c542-e503-40bd-b73c-5a9f37430834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.072675 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1ae4c542-e503-40bd-b73c-5a9f37430834 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:19.073896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1ae4c542-e503-40bd-b73c-5a9f37430834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:19.082604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1279/2609] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:19.090870 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dac6b7-643d-471d-9f45-943481f6c8a6 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:19.101304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dac6b7-643d-471d-9f45-943481f6c8a6 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:19.106100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:19.106745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.107029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.107304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.108002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:50:19.108309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:19.110560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:19.122443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1331/2610] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:50:19.127059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4c542-e503-40bd-b73c-5a9f37430834 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:19.136284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4c542-e503-40bd-b73c-5a9f37430834 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:19.141057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:19.143737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.143737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.143737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.143737 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:19.144791 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:19.256154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1280/2611] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:19 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:19.260488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:19.270859 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2079c-4361-4e00-a730-564c838f266e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:19.328179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:19.329423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.329863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.330163 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.330830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:19.332956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:19.444137 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1332/2612] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:19 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:19.448215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:19.459011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd98ea-6a81-48c9-bb19-b28bc4816408 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:19.463627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:19.464228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.464505 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.464805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.465488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:19.467432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:19.477967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1281/2613] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:19.482610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:19.492643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8bc2f-dfe9-40ef-9b58-b115a58e6f84 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:19.497293 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcaeb07-a1a0-421b-b26b-2469a83e7b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:19.497881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaeb07-a1a0-421b-b26b-2469a83e7b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.498188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaeb07-a1a0-421b-b26b-2469a83e7b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.498478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcaeb07-a1a0-421b-b26b-2469a83e7b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.499170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaeb07-a1a0-421b-b26b-2469a83e7b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:19.500998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcaeb07-a1a0-421b-b26b-2469a83e7b81 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:19.511686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1333/2614] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:19.516596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:19.526435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046675a-7414-4437-ab14-8702a31088be admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:19.530949 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:19.532362 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.532666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.532927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] PATH_INFO: `/v3/projects/e664288b45ae4b98ae22270ea7eac1e6/users/3023e29a392a4d97acbb11faecc0c9b8/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.548035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e664288b45ae4b98ae22270ea7eac1e6, user_id=3023e29a392a4d97acbb11faecc0c9b8, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:19.551401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:19.636410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1282/2615] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:19 2022] PUT /identity/v3/projects/e664288b45ae4b98ae22270ea7eac1e6/users/3023e29a392a4d97acbb11faecc0c9b8/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:19.642254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.642542 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.642814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.661817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:19.663049 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:19.732858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1334/2616] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:19.742765 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:19.744515 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:19.808658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9295f7e-9fa2-4d71-b383-5afa863f3c34 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:19.812389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205fe2-fa2f-4459-88ef-4dd5ffce6c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:19.813712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205fe2-fa2f-4459-88ef-4dd5ffce6c9f service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:19.813989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205fe2-fa2f-4459-88ef-4dd5ffce6c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:19.814385 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205fe2-fa2f-4459-88ef-4dd5ffce6c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:19.823889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205fe2-fa2f-4459-88ef-4dd5ffce6c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:19.825320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205fe2-fa2f-4459-88ef-4dd5ffce6c9f service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:19.835620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1283/2617] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:22.811837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:22.813401 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:22.827867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c845ad4-7c2b-406f-8d48-a41dc4edc1ad None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:22.834882 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:22.835637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:22.835989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:22.836336 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] PATH_INFO: `/v3/users/d50ca5da8075414fbd2e7aee04001320` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:22.837712 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d50ca5da8075414fbd2e7aee04001320)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:22.839913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:23.004644 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:23.018335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd50ca5da8075414fbd2e7aee04001320'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:23.032844 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd50ca5da8075414fbd2e7aee04001320'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:23.118932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd50ca5da8075414fbd2e7aee04001320'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:23.120078 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd50ca5da8075414fbd2e7aee04001320'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:23.120905 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd50ca5da8075414fbd2e7aee04001320'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:23.137100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1335/2618] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:50:22 2022] DELETE /identity/v3/users/d50ca5da8075414fbd2e7aee04001320 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:23.141996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:23.142532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:23.143092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:23.168923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:23.170182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:23.195728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:23.244416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1284/2619] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:23.255109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:23.257272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:23.337491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c877e39-942e-42a4-ac1c-7acdc9d1bce9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:23.342777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbaab3-b871-4ffb-82e8-566075bfb103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:23.344598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbaab3-b871-4ffb-82e8-566075bfb103 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:23.345081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbaab3-b871-4ffb-82e8-566075bfb103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:23.345565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbaab3-b871-4ffb-82e8-566075bfb103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:23.357304 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbaab3-b871-4ffb-82e8-566075bfb103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:23.359127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbaab3-b871-4ffb-82e8-566075bfb103 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:23.375622 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1336/2620] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:23.891403 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:23.893024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:23.936016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ca445-6459-4f8d-8170-f3739e789bab None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:23.943167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:23.944135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:23.944503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:23.944832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] PATH_INFO: `/v3/projects/e696cc6a3e7e4d939f874cc7c14df3f0` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:23.946245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e696cc6a3e7e4d939f874cc7c14df3f0)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:23.956204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:24.084005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e696cc6a3e7e4d939f874cc7c14df3f0', 'request_id': 'req-e04c8519-6141-45b4-8e8a-5883d7b749c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:24.091454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e696cc6a3e7e4d939f874cc7c14df3f0', 'request_id': 'req-e04c8519-6141-45b4-8e8a-5883d7b749c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:24.115012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e696cc6a3e7e4d939f874cc7c14df3f0', 'request_id': 'req-e04c8519-6141-45b4-8e8a-5883d7b749c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:24.197687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] The token cache is being invalidate because project e696cc6a3e7e4d939f874cc7c14df3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:50:24.199136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:24.199136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e04c8519-6141-45b4-8e8a-5883d7b749c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:24.200308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1285/2621] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:50:23 2022] DELETE /identity/v3/projects/e696cc6a3e7e4d939f874cc7c14df3f0 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:24.205455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4354907-ae52-43fb-93aa-538de46ec15f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.205924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4354907-ae52-43fb-93aa-538de46ec15f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.206052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4354907-ae52-43fb-93aa-538de46ec15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.230916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d4354907-ae52-43fb-93aa-538de46ec15f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:24.233700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d4354907-ae52-43fb-93aa-538de46ec15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:24.345473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1337/2622] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:24.353620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4ae4-96fc-497d-893c-4b4cd312ae37 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.354414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4ae4-96fc-497d-893c-4b4cd312ae37 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.354923 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b4ae4-96fc-497d-893c-4b4cd312ae37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.377105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1a3b4ae4-96fc-497d-893c-4b4cd312ae37 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:24.378572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1a3b4ae4-96fc-497d-893c-4b4cd312ae37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:24.386889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1286/2623] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:24.391516 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4354907-ae52-43fb-93aa-538de46ec15f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:24.401455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4354907-ae52-43fb-93aa-538de46ec15f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:24.406901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:24.407549 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.407932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.408182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.408879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:50:24.409182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:24.410816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:24.423504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1338/2624] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:50:24.427877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4ae4-96fc-497d-893c-4b4cd312ae37 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:24.437271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b4ae4-96fc-497d-893c-4b4cd312ae37 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:24.442268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:24.442875 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.443157 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.446098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.446098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:24.446098 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:24.581772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1287/2625] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:24 2022] POST /identity/v3/projects => generated 405 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:24.586473 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:24.609214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d0432-24ed-4125-92b8-a85ac7d6cb6b admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:24.667578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:24.695799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:24.695799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.695799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.695799 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.696119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:24.700295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:24.848610 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1339/2626] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:24 2022] POST /identity/v3/users => generated 303 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:24.849767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:24.861260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e931851-e867-471f-b6cb-e36486d93b5b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:24.866910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:24.867530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.868011 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.868289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.869149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:24.871317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:24.889104 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1288/2627] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:24.892783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:24.908255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82373753-4454-4b97-95b2-6895223e7017 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:24.920547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5da9f-df5e-46ab-bab7-17061ac71679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:24.921626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5da9f-df5e-46ab-bab7-17061ac71679 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.921881 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5da9f-df5e-46ab-bab7-17061ac71679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.922872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5da9f-df5e-46ab-bab7-17061ac71679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.923600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5da9f-df5e-46ab-bab7-17061ac71679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:24.925207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db5da9f-df5e-46ab-bab7-17061ac71679 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:24.942418 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:24.948484 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1340/2628] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:24.953117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2a19e-0f0e-4d18-8417-1f32f11f6fee admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:24.962896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:24.963847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:24.964407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:24.964860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] PATH_INFO: `/v3/projects/3f07bc53b5f04c21b99edf498f82a47f/users/4d6771e0720d4fc2b7716fc15364bc28/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:24.991569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f07bc53b5f04c21b99edf498f82a47f, user_id=4d6771e0720d4fc2b7716fc15364bc28, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:24.994427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:25.092928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1289/2629] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:24 2022] PUT /identity/v3/projects/3f07bc53b5f04c21b99edf498f82a47f/users/4d6771e0720d4fc2b7716fc15364bc28/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:25.096367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b91c3-d6a0-4018-805a-556341063261 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:25.096672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b91c3-d6a0-4018-805a-556341063261 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:25.096945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b91c3-d6a0-4018-805a-556341063261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:25.118173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-0b1b91c3-d6a0-4018-805a-556341063261 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:25.119408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0b1b91c3-d6a0-4018-805a-556341063261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:25.201761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1341/2630] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:25.213990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:25.216296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:25.311049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:25.312237 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78311d34-7e27-4a17-a08b-e6e901dc1457 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:25.324422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:25.325674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:25.326175 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:25.327342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:25.341646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:25.344596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:25.357862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1290/2631] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:25.690525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16340b6-6d97-4a47-aae4-316db61cb1a7 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:25.691007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16340b6-6d97-4a47-aae4-316db61cb1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:25.691261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16340b6-6d97-4a47-aae4-316db61cb1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:25.726194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-a16340b6-6d97-4a47-aae4-316db61cb1a7 None None] MFA Rules not processed for user `3023e29a392a4d97acbb11faecc0c9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:25.726194 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a16340b6-6d97-4a47-aae4-316db61cb1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:25.822820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1342/2632] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:25.827631 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:25.829225 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:25.844832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeec50f-eb7a-4427-988d-a07250a1af39 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:25.849837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:25.850488 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:25.850774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:25.851048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:25.851732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:25.853649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:25.946239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1291/2633] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:25 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:25.947586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16340b6-6d97-4a47-aae4-316db61cb1a7 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:25.959327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16340b6-6d97-4a47-aae4-316db61cb1a7 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:26.009348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:26.022848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:26.023572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:26.023987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:26.024347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:26.025075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:26.027042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:26.128902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1343/2634] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:25 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:26.132909 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:26.144329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fd42c-b240-4314-a2d9-157f68c1cc47 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:26.149017 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:26.149862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:26.150219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:26.150469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:26.151142 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:26.152712 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:26.163767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1292/2635] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:26.168053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:26.179573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3f7a7-ecfc-4096-a4f4-c614d6aab714 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:26.184320 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:26.185069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:26.185294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:26.185572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] PATH_INFO: `/v3/projects/f02877d9307944d78ebe93ea3d5b9dfb/users/984353383e614d638fbb0eb15cd944a2/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:26.201494 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02877d9307944d78ebe93ea3d5b9dfb, user_id=984353383e614d638fbb0eb15cd944a2, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:26.204225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:26.295182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1344/2636] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:26 2022] PUT /identity/v3/projects/f02877d9307944d78ebe93ea3d5b9dfb/users/984353383e614d638fbb0eb15cd944a2/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:26.297321 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:26.310992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e56979-9086-4b9d-9351-a41debb46451 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:26.373354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a67c1d-c79d-4b0a-8354-f726478d05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:26.373974 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67c1d-c79d-4b0a-8354-f726478d05b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:26.374296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67c1d-c79d-4b0a-8354-f726478d05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:26.374575 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a67c1d-c79d-4b0a-8354-f726478d05b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:26.375480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a67c1d-c79d-4b0a-8354-f726478d05b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:26.377070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a67c1d-c79d-4b0a-8354-f726478d05b8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:26.386939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1293/2637] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:26.391439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:26.401966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7c8d-c273-4ccf-97e1-ec2662b5b573 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:26.407284 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:26.408316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:26.408670 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:26.408864 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] PATH_INFO: `/v3/projects/f02877d9307944d78ebe93ea3d5b9dfb/users/984353383e614d638fbb0eb15cd944a2/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:26.410885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02877d9307944d78ebe93ea3d5b9dfb, user_id=984353383e614d638fbb0eb15cd944a2, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:26.413570 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:26.425738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1345/2638] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:26 2022] PUT /identity/v3/projects/f02877d9307944d78ebe93ea3d5b9dfb/users/984353383e614d638fbb0eb15cd944a2/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:26.863879 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2437d-4aa8-45fe-a9b5-f2f692d41589 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:26.864601 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2437d-4aa8-45fe-a9b5-f2f692d41589 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:26.864908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2437d-4aa8-45fe-a9b5-f2f692d41589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:26.912591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e9e2437d-4aa8-45fe-a9b5-f2f692d41589 None None] MFA Rules not processed for user `4d6771e0720d4fc2b7716fc15364bc28`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:26.914248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e9e2437d-4aa8-45fe-a9b5-f2f692d41589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:26.993194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1294/2639] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:26.997665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:27.060860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328212-e6ff-4e20-8ab5-0ed9691f9a1f admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:27.066930 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:27.067837 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:27.068214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:27.068583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:27.069555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:27.071850 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:27.182230 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1346/2640] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:26 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:27.186215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2437d-4aa8-45fe-a9b5-f2f692d41589 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:27.197567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2437d-4aa8-45fe-a9b5-f2f692d41589 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:27.245824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:27.262557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:27.263215 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:27.263499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:27.263783 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:27.264486 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:27.266980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:27.398440 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1295/2641] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:27 2022] POST /identity/v3/users => generated 308 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:27.399187 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:27.413560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4f2c0-8a60-476b-a66a-5719fb5db85a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:27.418332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:27.419228 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:27.419547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:27.419891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:27.421287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:27.423491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:27.439510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1347/2642] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:27.443539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:27.457434 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f8198-0a17-4af8-ad0c-3ba61ed9972b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:27.462589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:27.463365 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:27.469450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:27.469450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] PATH_INFO: `/v3/projects/6449960db8d7494c8a2e908615bfc135/users/8b909e3316884b2389579c46e76c2112/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:27.483808 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6449960db8d7494c8a2e908615bfc135, user_id=8b909e3316884b2389579c46e76c2112, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:27.486719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:27.609983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1296/2643] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:27 2022] PUT /identity/v3/projects/6449960db8d7494c8a2e908615bfc135/users/8b909e3316884b2389579c46e76c2112/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:50:27.614412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:27.625418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5883-0846-4ace-b01f-91b7c0ccf9fb admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:27.831962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b21ce-1d7c-4230-95d6-9d63922df1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:27.832874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b21ce-1d7c-4230-95d6-9d63922df1ac admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:27.832874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b21ce-1d7c-4230-95d6-9d63922df1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:27.833112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b21ce-1d7c-4230-95d6-9d63922df1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:27.833805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b21ce-1d7c-4230-95d6-9d63922df1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:27.835453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b21ce-1d7c-4230-95d6-9d63922df1ac admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:27.848203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1348/2644] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:27.852451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:27.863357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973b3fc-9550-45a4-aab1-d19e8369b338 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:27.867939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:27.868586 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:27.868865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:27.870167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] PATH_INFO: `/v3/projects/6449960db8d7494c8a2e908615bfc135/users/8b909e3316884b2389579c46e76c2112/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:27.871137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6449960db8d7494c8a2e908615bfc135, user_id=8b909e3316884b2389579c46e76c2112, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:27.873717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:27.884566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1297/2645] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:27 2022] PUT /identity/v3/projects/6449960db8d7494c8a2e908615bfc135/users/8b909e3316884b2389579c46e76c2112/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:29.928167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3ff83-aeb1-46a1-8813-2f75655b8e64 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:29.928700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3ff83-aeb1-46a1-8813-2f75655b8e64 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:29.928700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3ff83-aeb1-46a1-8813-2f75655b8e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:29.957643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:29.978323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-75a3ff83-aeb1-46a1-8813-2f75655b8e64 None None] MFA Rules not processed for user `8b909e3316884b2389579c46e76c2112`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:29.979701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-75a3ff83-aeb1-46a1-8813-2f75655b8e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:30.120537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1349/2646] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:32.348245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:32.350255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:32.439996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62eb9d-2d8d-40d0-8a39-624e3307d90e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:32.449343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:32.449992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:32.453511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:32.453511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:32.453511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:32.465794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:32.467746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:32.479428 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1298/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:32.562562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3ff83-aeb1-46a1-8813-2f75655b8e64 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:32.634422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3ff83-aeb1-46a1-8813-2f75655b8e64 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:32.638984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:32.639633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:32.639912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:32.640184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] PATH_INFO: `/v3/users/4d6771e0720d4fc2b7716fc15364bc28` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:32.642403 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d6771e0720d4fc2b7716fc15364bc28)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:32.644345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:32.862237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d6771e0720d4fc2b7716fc15364bc28'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:32.892277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d6771e0720d4fc2b7716fc15364bc28'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:32.969540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d6771e0720d4fc2b7716fc15364bc28'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:32.970296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d6771e0720d4fc2b7716fc15364bc28'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:32.971323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d6771e0720d4fc2b7716fc15364bc28'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:32.992545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1350/2648] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:50:32 2022] DELETE /identity/v3/users/4d6771e0720d4fc2b7716fc15364bc28 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:33.322929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:33.341050 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04aa35c-74b6-4ca0-b8d5-b659772f67f6 service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:33.418865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:33.419510 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:33.419789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:33.420061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] PATH_INFO: `/v3/projects/3f07bc53b5f04c21b99edf498f82a47f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:33.427282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f07bc53b5f04c21b99edf498f82a47f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:33.427464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:33.557550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:33.581134 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f07bc53b5f04c21b99edf498f82a47f', 'request_id': 'req-03e7e721-0cbe-4051-82d8-f81973434737'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:33.591085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f07bc53b5f04c21b99edf498f82a47f', 'request_id': 'req-03e7e721-0cbe-4051-82d8-f81973434737'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:33.599578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f07bc53b5f04c21b99edf498f82a47f', 'request_id': 'req-03e7e721-0cbe-4051-82d8-f81973434737'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:33.691111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] The token cache is being invalidate because project 3f07bc53b5f04c21b99edf498f82a47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:50:33.691111 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:33.692219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-03e7e721-0cbe-4051-82d8-f81973434737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:33.695352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1299/2649] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:50:33 2022] DELETE /identity/v3/projects/3f07bc53b5f04c21b99edf498f82a47f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:33.698266 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:33.700375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:33.714276 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c07cdd-804f-4bad-bacd-1b901fc64dbc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:33.726594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:33.761653 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:33.761937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:33.762251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:33.782022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:33.782777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:33.783159 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:33.783496 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] PATH_INFO: `/v3/users/8b909e3316884b2389579c46e76c2112` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:33.784972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b909e3316884b2389579c46e76c2112)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:33.786872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:33.795068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] MFA Rules not processed for user `984353383e614d638fbb0eb15cd944a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:33.797668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:33.873315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1300/2650] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:33.889447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:33.891397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:33.969833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88668ef4-1119-4cc6-a39f-7cfaf956e1f2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:33.972556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b909e3316884b2389579c46e76c2112'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:33.975682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:33.976383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:33.976736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:33.977016 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:33.978849 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:33.985211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b909e3316884b2389579c46e76c2112'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.014689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:34.037764 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b909e3316884b2389579c46e76c2112'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.039711 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:34.040244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b909e3316884b2389579c46e76c2112'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.041305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-ae38f946-7a85-4091-aa1d-b21db96bd0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b909e3316884b2389579c46e76c2112'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.041439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:34.052195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1301/2651] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:50:34.061594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1351/2652] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:50:33 2022] DELETE /identity/v3/users/8b909e3316884b2389579c46e76c2112 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:50:34.313932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:34.315545 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:34.340658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c40ac-0db5-4fcc-a9be-8a3e4fa05419 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:34.408149 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:34.408807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:34.409090 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:34.409367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] PATH_INFO: `/v3/projects/6449960db8d7494c8a2e908615bfc135` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:34.410560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6449960db8d7494c8a2e908615bfc135)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:34.412557 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:34.542818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6449960db8d7494c8a2e908615bfc135', 'request_id': 'req-2e724f38-ed45-49a6-8296-7e41c4f4cb14'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.558602 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6449960db8d7494c8a2e908615bfc135', 'request_id': 'req-2e724f38-ed45-49a6-8296-7e41c4f4cb14'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.565349 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6449960db8d7494c8a2e908615bfc135', 'request_id': 'req-2e724f38-ed45-49a6-8296-7e41c4f4cb14'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.638542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] The token cache is being invalidate because project 6449960db8d7494c8a2e908615bfc135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:50:34.638827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.639497 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e724f38-ed45-49a6-8296-7e41c4f4cb14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:50:34.641877 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1302/2653] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:50:34 2022] DELETE /identity/v3/projects/6449960db8d7494c8a2e908615bfc135 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:34.647157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642bfae-0e6a-45da-a5c6-abea7035923c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:34.647427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642bfae-0e6a-45da-a5c6-abea7035923c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:34.647704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642bfae-0e6a-45da-a5c6-abea7035923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:34.668919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b642bfae-0e6a-45da-a5c6-abea7035923c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:34.670127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b642bfae-0e6a-45da-a5c6-abea7035923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:34.750358 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1352/2654] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:34.754377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567103f1-c504-4011-8550-c2988c42b5db None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:34.754847 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567103f1-c504-4011-8550-c2988c42b5db None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:34.755895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567103f1-c504-4011-8550-c2988c42b5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:34.779869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-567103f1-c504-4011-8550-c2988c42b5db None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:34.781324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-567103f1-c504-4011-8550-c2988c42b5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:34.789769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1303/2655] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:50:34.794111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642bfae-0e6a-45da-a5c6-abea7035923c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:34.803965 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642bfae-0e6a-45da-a5c6-abea7035923c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:34.809001 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:34.809626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:34.809990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:34.810356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:34.811120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:50:34.811517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:34.813902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:34.826212 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1353/2656] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:50:34.835703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567103f1-c504-4011-8550-c2988c42b5db None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:34.847307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567103f1-c504-4011-8550-c2988c42b5db None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:34.852427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:34.853164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:34.853529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:34.853896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:34.854773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:34.856797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:34.963568 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1304/2657] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:34 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:50:34.964794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:34.978283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fd98a-f05d-4491-a87a-12197f1989a7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:35.000514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:35.039221 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:35.039838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.040112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.040378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.041024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:35.042939 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:35.141973 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1354/2658] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:34 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:35.145464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:35.157433 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464c2b1-4115-40be-957c-48fd93909f65 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:35.164624 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:35.165389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.170770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.170770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.170770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:35.170770 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:35.184075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1305/2659] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:35.185309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:35.201239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e3bd5-c363-4747-88ce-22e5c9a705bc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:35.207033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5218e-e56b-49d8-afd2-d4bea4a8e42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:35.207784 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5218e-e56b-49d8-afd2-d4bea4a8e42d admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.208143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5218e-e56b-49d8-afd2-d4bea4a8e42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.208487 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5218e-e56b-49d8-afd2-d4bea4a8e42d admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.209275 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5218e-e56b-49d8-afd2-d4bea4a8e42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:35.211099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5218e-e56b-49d8-afd2-d4bea4a8e42d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:35.237442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1355/2660] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:35.241501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:35.255667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d70eca-f8ed-420f-b79b-dd2b5a282dd6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:35.264047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:35.265824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.282083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.282083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] PATH_INFO: `/v3/projects/a6408f976dc1401e976cbdc4fa9502e4/users/2dab31bfa8b0463c94d590d2c80b1bca/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.286057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6408f976dc1401e976cbdc4fa9502e4, user_id=2dab31bfa8b0463c94d590d2c80b1bca, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:35.288846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:35.357013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1306/2661] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:35 2022] PUT /identity/v3/projects/a6408f976dc1401e976cbdc4fa9502e4/users/2dab31bfa8b0463c94d590d2c80b1bca/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:35.368635 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64577b40-170c-4c0a-8e6c-c5ddc07240e5 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.368921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64577b40-170c-4c0a-8e6c-c5ddc07240e5 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.369196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64577b40-170c-4c0a-8e6c-c5ddc07240e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.405335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-64577b40-170c-4c0a-8e6c-c5ddc07240e5 None None] MFA Rules not processed for user `2dab31bfa8b0463c94d590d2c80b1bca`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:35.407119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-64577b40-170c-4c0a-8e6c-c5ddc07240e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:35.494186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1356/2662] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:35.498903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:35.506076 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:35.584961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d82e4-70e7-417b-8ca0-d8d90a714e1b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:35.590001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:35.590691 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.590969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.591240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.591994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:35.594824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:35.728395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1307/2663] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:50:35 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:35.732161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64577b40-170c-4c0a-8e6c-c5ddc07240e5 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:35.747007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64577b40-170c-4c0a-8e6c-c5ddc07240e5 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:35.807989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:35.808692 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.808877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.809150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.809831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:35.811836 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:35.963156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1357/2664] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:50:35 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:50:35.982102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:35.982102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa113e-5d31-4c6b-8228-a16c43f296db admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:35.987982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:35.988741 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:35.989044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:35.989695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:35.990627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:35.993788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:36.010790 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1308/2665] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:36.012088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:36.024208 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daaa68-b757-4ae5-95c4-fd8ec05d6b1c admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:36.029950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:36.040705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:36.040705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:36.040705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] PATH_INFO: `/v3/projects/32796d938707460a9a12104a204eacbe/users/f1f9996a764043f6b87b9564f5c00efa/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:36.071794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32796d938707460a9a12104a204eacbe, user_id=f1f9996a764043f6b87b9564f5c00efa, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:36.074681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:36.078693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:36.185584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1358/2666] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:36 2022] PUT /identity/v3/projects/32796d938707460a9a12104a204eacbe/users/f1f9996a764043f6b87b9564f5c00efa/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:36.190698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:36.202312 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64d7b1-3cf1-4a9a-837c-127d13e1da40 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:36.264757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d317f-3331-47c4-a42a-486de1a64b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:36.266009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d317f-3331-47c4-a42a-486de1a64b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:36.266465 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d317f-3331-47c4-a42a-486de1a64b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:36.266784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d317f-3331-47c4-a42a-486de1a64b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:36.267480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d317f-3331-47c4-a42a-486de1a64b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:36.269097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d317f-3331-47c4-a42a-486de1a64b76 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:36.283205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1309/2667] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:50:36.287580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:36.314716 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336823c4-53f3-42b7-917b-f35eb0f7c7e5 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:36.320755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:36.321476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:36.321954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:36.322195 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] PATH_INFO: `/v3/projects/32796d938707460a9a12104a204eacbe/users/f1f9996a764043f6b87b9564f5c00efa/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:36.324974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32796d938707460a9a12104a204eacbe, user_id=f1f9996a764043f6b87b9564f5c00efa, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:36.328440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:36.342869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1359/2668] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:50:36 2022] PUT /identity/v3/projects/32796d938707460a9a12104a204eacbe/users/f1f9996a764043f6b87b9564f5c00efa/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:50:36.355029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:36.355605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:36.356319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:36.388795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] MFA Rules not processed for user `f1f9996a764043f6b87b9564f5c00efa`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:50:36.390036 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:36.475274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:36.481507 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1310/2669] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:50:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:50:36.495651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:36.497490 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:36.574362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c654-3fb5-4cf2-97e6-382f641d2d8d admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:36.578805 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:36.584479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:36.584479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:36.584479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:36.644145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:36.644145 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:36.658369 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1360/2670] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:46.650789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:46.820395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:50:47.128067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:50:47.129517 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:50:47.141745 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed1e3-0337-4a52-8694-8b1c2749d1c9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:50:47.145574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27c0f0-4d9f-4239-b980-0dd74ebb788e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:50:47.146805 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c0f0-4d9f-4239-b980-0dd74ebb788e service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:50:47.147097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c0f0-4d9f-4239-b980-0dd74ebb788e service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:50:47.147379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27c0f0-4d9f-4239-b980-0dd74ebb788e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:50:47.156992 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27c0f0-4d9f-4239-b980-0dd74ebb788e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:50:47.158524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27c0f0-4d9f-4239-b980-0dd74ebb788e service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:50:47.169814 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1311/2671] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:50:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:50:57.164627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:08.764008 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:08.765510 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:08.831454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8007f-0f02-43a7-acbd-fb83ad66fc02 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:08.836093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:08.839499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:08.839499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:08.839499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] PATH_INFO: `/v3/users/06129f511d624d8a97816e84c0b83c59` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:08.839499 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06129f511d624d8a97816e84c0b83c59)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:08.840343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:09.015282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06129f511d624d8a97816e84c0b83c59'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.027801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06129f511d624d8a97816e84c0b83c59'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.082795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06129f511d624d8a97816e84c0b83c59'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.083362 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06129f511d624d8a97816e84c0b83c59'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.083963 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06129f511d624d8a97816e84c0b83c59'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.097239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1361/2672] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:08 2022] DELETE /identity/v3/users/06129f511d624d8a97816e84c0b83c59 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:09.101667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:09.102187 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:09.102301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:09.121373 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:09.122619 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:09.181083 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1312/2673] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:09.191146 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:09.192531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:09.249669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f845-8f58-478b-b188-70239d1da1d4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:09.254115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:09.254752 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:09.255034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:09.255307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:09.264099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:09.265525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:09.275750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1362/2674] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:09.643548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:09.645014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:09.657063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fd6d2-706d-493f-91d6-61baa4c7965e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:09.661862 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:09.662609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:09.667521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:09.667521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] PATH_INFO: `/v3/projects/50808a876ea24416b631789b73465f7f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:09.667521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50808a876ea24416b631789b73465f7f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:09.667521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:09.766668 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50808a876ea24416b631789b73465f7f', 'request_id': 'req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.772188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50808a876ea24416b631789b73465f7f', 'request_id': 'req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.778255 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50808a876ea24416b631789b73465f7f', 'request_id': 'req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.863457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] The token cache is being invalidate because project 50808a876ea24416b631789b73465f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:09.863742 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.864367 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:09.865896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1313/2675] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:09 2022] DELETE /identity/v3/projects/50808a876ea24416b631789b73465f7f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:09.869860 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:09.879903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec624009-1e09-4d7d-8059-fda43716adc2 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:09.897865 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:09.935502 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:09.936134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:09.936409 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:09.936679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] PATH_INFO: `/v3/users/4af6a0aa99a1419581c69e70c0046cf0` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:09.937817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af6a0aa99a1419581c69e70c0046cf0)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:09.939862 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:10.180953 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af6a0aa99a1419581c69e70c0046cf0'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.192624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af6a0aa99a1419581c69e70c0046cf0'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.270749 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af6a0aa99a1419581c69e70c0046cf0'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.271485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af6a0aa99a1419581c69e70c0046cf0'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.272069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-4a48ed6e-549c-4c44-93ac-4655437a3852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af6a0aa99a1419581c69e70c0046cf0'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.285141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1363/2676] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:09 2022] DELETE /identity/v3/users/4af6a0aa99a1419581c69e70c0046cf0 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:10.660516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:10.663212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:10.675908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5af0f-dd3e-4d42-81fa-c7cd4f65ded4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:10.733068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:10.733707 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:10.733982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:10.734289 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] PATH_INFO: `/v3/projects/52c5bed6e26047959a967c87d3fcb9ce` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:10.735426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c5bed6e26047959a967c87d3fcb9ce)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:10.737419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:10.821324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:10.839409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c5bed6e26047959a967c87d3fcb9ce', 'request_id': 'req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.845337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c5bed6e26047959a967c87d3fcb9ce', 'request_id': 'req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.853803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c5bed6e26047959a967c87d3fcb9ce', 'request_id': 'req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.928512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] The token cache is being invalidate because project 52c5bed6e26047959a967c87d3fcb9ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:10.928803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.929482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f0f27cb1-d9ab-402a-92e5-e1a4ccd3f3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:10.931020 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1314/2677] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:10 2022] DELETE /identity/v3/projects/52c5bed6e26047959a967c87d3fcb9ce => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:10.935943 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d652f6-926c-4eb4-9abe-d841c03ef7d1 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:10.936514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d652f6-926c-4eb4-9abe-d841c03ef7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:10.936514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d652f6-926c-4eb4-9abe-d841c03ef7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:10.957130 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e6d652f6-926c-4eb4-9abe-d841c03ef7d1 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:10.958378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e6d652f6-926c-4eb4-9abe-d841c03ef7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:11.021082 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1364/2678] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:11.030407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea87c7-9959-4e2a-9c49-0b459257b6f2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.030692 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea87c7-9959-4e2a-9c49-0b459257b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.031015 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea87c7-9959-4e2a-9c49-0b459257b6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.052656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-98ea87c7-9959-4e2a-9c49-0b459257b6f2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:11.053882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-98ea87c7-9959-4e2a-9c49-0b459257b6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:11.061840 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1315/2679] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:11.066904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d652f6-926c-4eb4-9abe-d841c03ef7d1 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.076313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d652f6-926c-4eb4-9abe-d841c03ef7d1 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.080815 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.081412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.081687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.081955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.082674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:51:11.082974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.084544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:11.096547 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1365/2680] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:51:11.100806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea87c7-9959-4e2a-9c49-0b459257b6f2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.109798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea87c7-9959-4e2a-9c49-0b459257b6f2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.114968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.115566 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.115843 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.116186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.116845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.119039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:11.198736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1316/2681] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:51:11 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:51:11.202651 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.212395 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a3f86-3ded-4d86-85c2-7602bb4d2111 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.238428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:11.267855 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.268462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.268737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.269009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.269668 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.271608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:11.351084 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1366/2682] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:51:11 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:11.356167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.367665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5293-5487-4d32-a1bc-2df649a99b8d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.372550 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52707373-2549-4369-9700-91587000beb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.373162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707373-2549-4369-9700-91587000beb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.373441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707373-2549-4369-9700-91587000beb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.373713 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707373-2549-4369-9700-91587000beb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.374447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52707373-2549-4369-9700-91587000beb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.376033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52707373-2549-4369-9700-91587000beb7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:11.386818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1317/2683] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:11.390762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.400821 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f849-077a-410c-b5f7-49e3eafe6343 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.405485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc8838-9812-4e59-b9cb-b4f8945ee049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.406083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc8838-9812-4e59-b9cb-b4f8945ee049 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.406432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc8838-9812-4e59-b9cb-b4f8945ee049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.406705 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc8838-9812-4e59-b9cb-b4f8945ee049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.407394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc8838-9812-4e59-b9cb-b4f8945ee049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.409515 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc8838-9812-4e59-b9cb-b4f8945ee049 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:11.420231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1367/2684] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:11.425007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52707373-2549-4369-9700-91587000beb7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.434845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52707373-2549-4369-9700-91587000beb7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.439413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.440572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.440845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.441119 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] PATH_INFO: `/v3/projects/0414481f0cd4423da82917f6f62e9481/users/6dd3f7af3ed042beb6a51034578a38c5/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.458471 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0414481f0cd4423da82917f6f62e9481, user_id=6dd3f7af3ed042beb6a51034578a38c5, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.461024 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:11.544890 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1318/2685] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:11 2022] PUT /identity/v3/projects/0414481f0cd4423da82917f6f62e9481/users/6dd3f7af3ed042beb6a51034578a38c5/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:11.557897 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55974-99be-42a9-8941-c0d8870b67aa None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.558211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55974-99be-42a9-8941-c0d8870b67aa None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.558489 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a55974-99be-42a9-8941-c0d8870b67aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.591637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c5a55974-99be-42a9-8941-c0d8870b67aa None None] MFA Rules not processed for user `6dd3f7af3ed042beb6a51034578a38c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:11.593119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c5a55974-99be-42a9-8941-c0d8870b67aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:11.668113 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1368/2686] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:11.672593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.743819 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283f456-8cb6-4175-8977-e4f28a12acfe admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.748501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.749121 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.749408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.749682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.750399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.753056 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:11.833031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1319/2687] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:51:11 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:11.837106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55974-99be-42a9-8941-c0d8870b67aa None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:11.848131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a55974-99be-42a9-8941-c0d8870b67aa None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:11.910420 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:11.911038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:11.911328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:11.911601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:11.912265 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:11.914193 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:12.020686 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1369/2688] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:51:11 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:12.024700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:12.035989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e50c2f-d0cb-401a-93fb-9e06ed6683e5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:12.040789 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:12.041407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:12.041688 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:12.041963 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:12.042706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:12.044300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:12.056491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1320/2689] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:12.061439 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:12.072744 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97721283-9956-4752-b8f2-c1f7ed2024a6 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:12.077849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:12.078539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:12.078817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:12.079120 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] PATH_INFO: `/v3/projects/2e924d9093be423fbbe4ee63ca430948/users/a24a8e4a36474371bead2073b82e30d9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:12.094131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e924d9093be423fbbe4ee63ca430948, user_id=a24a8e4a36474371bead2073b82e30d9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:12.097341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:12.168550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1370/2690] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:12 2022] PUT /identity/v3/projects/2e924d9093be423fbbe4ee63ca430948/users/a24a8e4a36474371bead2073b82e30d9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:12.177902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:12.183298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89859060-f417-4c70-9d8e-c006fcb9b68c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:12.202582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:12.240582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5fda6f-cf2c-457f-b161-9015980f4821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:12.241194 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fda6f-cf2c-457f-b161-9015980f4821 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:12.241472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fda6f-cf2c-457f-b161-9015980f4821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:12.241744 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5fda6f-cf2c-457f-b161-9015980f4821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:12.242469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5fda6f-cf2c-457f-b161-9015980f4821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:12.244044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5fda6f-cf2c-457f-b161-9015980f4821 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:12.253794 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1321/2691] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:51:12.258046 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:12.263429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:12.268792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef61e9-4dc7-46f4-983f-ba89d7e1d8b1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:12.273432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:12.274026 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:12.274331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:12.274603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] PATH_INFO: `/v3/projects/2e924d9093be423fbbe4ee63ca430948/users/a24a8e4a36474371bead2073b82e30d9/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:12.276518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e924d9093be423fbbe4ee63ca430948, user_id=a24a8e4a36474371bead2073b82e30d9, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:12.279169 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:12.289730 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1371/2692] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:12 2022] PUT /identity/v3/projects/2e924d9093be423fbbe4ee63ca430948/users/a24a8e4a36474371bead2073b82e30d9/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:51:12.300417 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a94945-6bd6-4be8-a01f-a425f1900a30 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:12.300704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a94945-6bd6-4be8-a01f-a425f1900a30 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:12.300988 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a94945-6bd6-4be8-a01f-a425f1900a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:12.329153 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f1a94945-6bd6-4be8-a01f-a425f1900a30 None None] MFA Rules not processed for user `a24a8e4a36474371bead2073b82e30d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:12.330582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f1a94945-6bd6-4be8-a01f-a425f1900a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:12.399702 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1322/2693] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:12.411466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:12.413160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:12.472097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cb9a5-5421-4f76-9b18-31ee2efe9377 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:12.475785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:12.476477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:12.476755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:12.477024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:12.486259 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:12.487856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:12.497354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1372/2694] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:12.695414 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a94945-6bd6-4be8-a01f-a425f1900a30 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:12.704941 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a94945-6bd6-4be8-a01f-a425f1900a30 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:12.709013 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:12.709618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:12.709935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:12.710214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:12.765234 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:12.766689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:12.776012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1323/2695] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:21.184890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:21.186506 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:21.248199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1f0e-9f8c-475d-821a-94815fc15d62 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:21.252431 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54734c-d487-4dbf-ab7e-c446a27472e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:21.253090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54734c-d487-4dbf-ab7e-c446a27472e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:21.253361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54734c-d487-4dbf-ab7e-c446a27472e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:21.253629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54734c-d487-4dbf-ab7e-c446a27472e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:21.255421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7e54734c-d487-4dbf-ab7e-c446a27472e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:21.327268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54734c-d487-4dbf-ab7e-c446a27472e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:21.328718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54734c-d487-4dbf-ab7e-c446a27472e4 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:21.338791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1373/2696] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:22.771971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=80 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:23.003514 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:23.013455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e94396-7fc6-4b5f-bd17-b9d099d11540 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:23.017102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:23.019057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:23.019338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:23.019912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:23.021493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:23.032239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:23.034343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:23.043996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1324/2697] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:25.346367 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:25.346929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:25.346929 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:25.366718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] MFA Rules not processed for user `3023e29a392a4d97acbb11faecc0c9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:25.367942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:25.418490 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:25.419264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:25.435488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1374/2698] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:25.504295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fb1cd-a98a-4326-b2f4-35f6c5a79b1a service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:25.511910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:25.512954 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:25.513344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:25.513627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] PATH_INFO: `/v3/users/6dd3f7af3ed042beb6a51034578a38c5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:25.514918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd3f7af3ed042beb6a51034578a38c5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:25.516823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:25.667222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:25.669075 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:25.735683 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd3f7af3ed042beb6a51034578a38c5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:25.758117 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fa3a9-00a6-472e-94fd-b382bdbdf3a8 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:25.767378 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:25.768060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:25.768345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:25.768617 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:25.770509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:25.775027 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd3f7af3ed042beb6a51034578a38c5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:25.787787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:25.789478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:25.802333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1375/2699] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:51:25.824607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:25.835562 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1620ca4-ece3-4fb6-b88d-75c9ae3cfcab service nova] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:25.840607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab51cc5-30c8-4740-bde8-f60e34401087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:25.840607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51cc5-30c8-4740-bde8-f60e34401087 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:25.840607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51cc5-30c8-4740-bde8-f60e34401087 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:25.840607 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab51cc5-30c8-4740-bde8-f60e34401087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:25.843116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2ab51cc5-30c8-4740-bde8-f60e34401087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:25.854246 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab51cc5-30c8-4740-bde8-f60e34401087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:25.856253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab51cc5-30c8-4740-bde8-f60e34401087 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:25.866258 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1376/2700] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:51:25.871232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd3f7af3ed042beb6a51034578a38c5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:25.881246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd3f7af3ed042beb6a51034578a38c5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:25.882089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd3f7af3ed042beb6a51034578a38c5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:25.890743 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1325/2701] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:25 2022] DELETE /identity/v3/users/6dd3f7af3ed042beb6a51034578a38c5 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:25.894998 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:25.897483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:25.897483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:25.916271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:25.917572 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:26.019868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1377/2702] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:26.029857 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:26.031837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:26.123323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de184c-7bef-4b9b-9981-d78133575f7c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:26.127721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:26.128354 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:26.128728 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:26.128908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:26.139670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:26.141172 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:26.156088 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1326/2703] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:51:26.714377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:26.716300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:26.734218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafbc96-bbc8-46ef-8b02-18bc82b2f94e None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:26.740838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:26.741612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:26.741961 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:26.743006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] PATH_INFO: `/v3/projects/0414481f0cd4423da82917f6f62e9481` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:26.744775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0414481f0cd4423da82917f6f62e9481)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:26.747254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:26.860157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0414481f0cd4423da82917f6f62e9481', 'request_id': 'req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:26.865887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0414481f0cd4423da82917f6f62e9481', 'request_id': 'req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:26.870808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:26.938603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:26.940499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:26.952208 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:27.013530 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91868bd-2c0a-4e78-82ec-3df9618a2334 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:27.018262 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0414481f0cd4423da82917f6f62e9481', 'request_id': 'req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.033437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] The token cache is being invalidate because project 0414481f0cd4423da82917f6f62e9481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:27.033866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.034740 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.036665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1378/2704] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:26 2022] DELETE /identity/v3/projects/0414481f0cd4423da82917f6f62e9481 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:27.040268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:27.041657 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:27.084133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:27.084133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:27.084133 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:27.084547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:27.086330 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:27.103962 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:27.105626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:27.115290 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf025eb9-c1fd-49c5-85c9-8b5a13887bf1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:27.116768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1327/2705] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:27.121135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:27.126029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:27.126029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:27.126029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] PATH_INFO: `/v3/users/a24a8e4a36474371bead2073b82e30d9` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:27.126029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24a8e4a36474371bead2073b82e30d9)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:27.126029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:27.363792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24a8e4a36474371bead2073b82e30d9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.379550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24a8e4a36474371bead2073b82e30d9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.504908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24a8e4a36474371bead2073b82e30d9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.505553 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24a8e4a36474371bead2073b82e30d9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.506214 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24a8e4a36474371bead2073b82e30d9'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:27.519687 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1379/2706] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:27 2022] DELETE /identity/v3/users/a24a8e4a36474371bead2073b82e30d9 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:51:27.815439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:27.816881 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:27.892272 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c98c91-020b-44c9-9a75-685102a498a3 service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:27.964386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feea078-3b18-4220-8991-7cdd75d012d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:27.965420 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feea078-3b18-4220-8991-7cdd75d012d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:27.965904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feea078-3b18-4220-8991-7cdd75d012d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:27.966451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feea078-3b18-4220-8991-7cdd75d012d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:27.972387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6feea078-3b18-4220-8991-7cdd75d012d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:27.985593 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:28.054972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feea078-3b18-4220-8991-7cdd75d012d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:28.056690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feea078-3b18-4220-8991-7cdd75d012d6 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:28.071781 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1328/2707] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:28.129411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:28.130928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:28.144520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e3a1-7fdc-486b-9b24-3541a2a789e4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:28.149114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:28.149762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.150048 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.150356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] PATH_INFO: `/v3/projects/2e924d9093be423fbbe4ee63ca430948` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.156652 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e924d9093be423fbbe4ee63ca430948)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:28.158874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:28.272568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e924d9093be423fbbe4ee63ca430948', 'request_id': 'req-8f09b5e3-16e0-4f1d-872b-a52505ed100c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:28.278100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e924d9093be423fbbe4ee63ca430948', 'request_id': 'req-8f09b5e3-16e0-4f1d-872b-a52505ed100c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:28.280634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:INSERT=3,keystone:DELETE=17,keystone:SELECT=80 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:28.389830 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e924d9093be423fbbe4ee63ca430948', 'request_id': 'req-8f09b5e3-16e0-4f1d-872b-a52505ed100c'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:28.403834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] The token cache is being invalidate because project 2e924d9093be423fbbe4ee63ca430948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:28.404118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:28.404885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-8f09b5e3-16e0-4f1d-872b-a52505ed100c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:28.407467 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1380/2708] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:28 2022] DELETE /identity/v3/projects/2e924d9093be423fbbe4ee63ca430948 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:28.413346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f6709-0eed-4de8-b2c3-ffe491fade76 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.413632 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f6709-0eed-4de8-b2c3-ffe491fade76 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.413906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f6709-0eed-4de8-b2c3-ffe491fade76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.436576 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c56f6709-0eed-4de8-b2c3-ffe491fade76 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:28.437798 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c56f6709-0eed-4de8-b2c3-ffe491fade76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:28.521504 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1329/2709] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:28.524254 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fb191-cbf3-48bb-9fd6-f4569476116c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.524751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fb191-cbf3-48bb-9fd6-f4569476116c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.525121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fb191-cbf3-48bb-9fd6-f4569476116c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.557904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-fb8fb191-cbf3-48bb-9fd6-f4569476116c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:28.560015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fb8fb191-cbf3-48bb-9fd6-f4569476116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:28.572407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1381/2710] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:28.576667 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f6709-0eed-4de8-b2c3-ffe491fade76 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:28.587274 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f6709-0eed-4de8-b2c3-ffe491fade76 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:28.592828 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:28.593464 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.593751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.594022 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.594757 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:51:28.595060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:28.596727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:28.610580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1330/2711] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:51:28.616054 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fb191-cbf3-48bb-9fd6-f4569476116c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:28.630928 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fb191-cbf3-48bb-9fd6-f4569476116c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:28.635636 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:28.636261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.636540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.636809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.637476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:28.639404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:28.755085 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1382/2712] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:51:28 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:28.759300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:28.770262 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625615-9cd2-47e4-a1cd-386c9f6794b1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:28.829654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:28.830506 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.831421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.831421 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.832078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:28.834400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:28.923033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1331/2713] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:51:28 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:28.928422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:28.939718 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0502e9-d3c8-4dfb-a353-8cf70ce423c1 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:28.957198 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:28.957853 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.958138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.958442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.959155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:28.960795 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:28.974698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1383/2714] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:51:28.979007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:28.991039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81a00-b9d3-4da9-a009-35edbf1ba9ce admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:28.996491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:28.997107 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:28.997384 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:28.997657 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:28.998380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:28.999966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:29.012301 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1332/2715] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:51:29.017540 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:29.028993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b546b06-3a2c-44e7-9cd5-7e2a2f4dad24 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:29.034244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b7036-f8c5-43d4-8bb4-bf4fa6c5de92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:29.034947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b7036-f8c5-43d4-8bb4-bf4fa6c5de92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:29.035251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b7036-f8c5-43d4-8bb4-bf4fa6c5de92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:29.035531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b7036-f8c5-43d4-8bb4-bf4fa6c5de92 admin admin] PATH_INFO: `/v3/projects/0415bf840a6f452d995aecae583eaa07/users/8938189deb9549579aa69b7518e93edd/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:29.037804 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:29.040630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:29.055810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55d438-63d3-4d16-a53d-af696aaa0c93 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:29.056052 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b7036-f8c5-43d4-8bb4-bf4fa6c5de92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0415bf840a6f452d995aecae583eaa07, user_id=8938189deb9549579aa69b7518e93edd, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:29.058640 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b7036-f8c5-43d4-8bb4-bf4fa6c5de92 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:29.060979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:29.061617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:29.061894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:29.062217 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] PATH_INFO: `/v3/users/2dab31bfa8b0463c94d590d2c80b1bca` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:29.063654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dab31bfa8b0463c94d590d2c80b1bca)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:29.066454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:29.073785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:29.176527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1384/2716] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:29 2022] PUT /identity/v3/projects/0415bf840a6f452d995aecae583eaa07/users/8938189deb9549579aa69b7518e93edd/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:29.187566 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:29.188002 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:29.188427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:29.228352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] MFA Rules not processed for user `8938189deb9549579aa69b7518e93edd`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:29.229582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:29.273146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dab31bfa8b0463c94d590d2c80b1bca'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:29.309769 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dab31bfa8b0463c94d590d2c80b1bca'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:29.334871 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1385/2717] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:51:29.345088 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:29.351918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:29.439790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc38db8-a4e6-4028-8857-72ea9d36fa14 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:29.446463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae718b-4bbb-4464-ae59-3ad84d4a263f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:29.447785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae718b-4bbb-4464-ae59-3ad84d4a263f service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:29.454642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae718b-4bbb-4464-ae59-3ad84d4a263f service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:29.454642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae718b-4bbb-4464-ae59-3ad84d4a263f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:29.461987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae718b-4bbb-4464-ae59-3ad84d4a263f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:29.463835 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae718b-4bbb-4464-ae59-3ad84d4a263f service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:29.477484 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dab31bfa8b0463c94d590d2c80b1bca'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:29.490478 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1386/2718] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:51:29.494774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dab31bfa8b0463c94d590d2c80b1bca'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:29.496043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dab31bfa8b0463c94d590d2c80b1bca'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:29.511773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1333/2719] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:29 2022] DELETE /identity/v3/users/2dab31bfa8b0463c94d590d2c80b1bca => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:29.515539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:29.518114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:29.518114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:29.542509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:29.544554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:29.566594 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:29.641684 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1387/2720] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:29.657609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:29.660754 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:29.756939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c957-0319-4c47-a3d5-cc6fed68eccc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:29.761248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:29.761920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:29.762352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:29.762830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:29.775085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:29.776570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:29.790288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1334/2721] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:30.618338 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:30.620033 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:30.655512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672c3a6-e9e3-4bc2-a39d-5cd0b6e955bc None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:30.655512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:30.655512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:30.655512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:30.656783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] PATH_INFO: `/v3/projects/a6408f976dc1401e976cbdc4fa9502e4` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:30.656783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6408f976dc1401e976cbdc4fa9502e4)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:30.656783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:30.844442 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6408f976dc1401e976cbdc4fa9502e4', 'request_id': 'req-69bd1aa8-97ad-429c-824d-214f81d45291'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:30.853430 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6408f976dc1401e976cbdc4fa9502e4', 'request_id': 'req-69bd1aa8-97ad-429c-824d-214f81d45291'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:30.951018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6408f976dc1401e976cbdc4fa9502e4', 'request_id': 'req-69bd1aa8-97ad-429c-824d-214f81d45291'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:30.971867 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] The token cache is being invalidate because project a6408f976dc1401e976cbdc4fa9502e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:30.972509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:30.974177 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:30.976277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1388/2722] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:30 2022] DELETE /identity/v3/projects/a6408f976dc1401e976cbdc4fa9502e4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:30.986448 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:30.992542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71630-0402-4eba-a052-a84d884dd23b service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:31.074260 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:31.075290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:31.075779 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:31.076258 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] PATH_INFO: `/v3/users/f1f9996a764043f6b87b9564f5c00efa` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:31.077562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f9996a764043f6b87b9564f5c00efa)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:31.080687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:31.294831 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:31.328389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f9996a764043f6b87b9564f5c00efa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:31.364800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f9996a764043f6b87b9564f5c00efa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:31.523169 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f9996a764043f6b87b9564f5c00efa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:31.531670 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f9996a764043f6b87b9564f5c00efa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:31.532944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2d2ebf91-834c-42ea-9511-e35826b2b5ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f9996a764043f6b87b9564f5c00efa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:31.539932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1335/2723] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:30 2022] DELETE /identity/v3/users/f1f9996a764043f6b87b9564f5c00efa => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:32.258488 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:32.259779 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:32.305719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1aa8-97ad-429c-824d-214f81d45291 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:32.473967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:32.474846 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:32.475135 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:32.475414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] PATH_INFO: `/v3/projects/32796d938707460a9a12104a204eacbe` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:32.478665 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32796d938707460a9a12104a204eacbe)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:32.482888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:32.772664 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:32.778792 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32796d938707460a9a12104a204eacbe', 'request_id': 'req-27a3f356-be2c-4f04-a9df-625ca443c702'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:32.807567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32796d938707460a9a12104a204eacbe', 'request_id': 'req-27a3f356-be2c-4f04-a9df-625ca443c702'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:32.887681 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32796d938707460a9a12104a204eacbe', 'request_id': 'req-27a3f356-be2c-4f04-a9df-625ca443c702'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:32.912702 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] The token cache is being invalidate because project 32796d938707460a9a12104a204eacbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:32.913103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:32.913701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-27a3f356-be2c-4f04-a9df-625ca443c702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:32.918791 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1389/2724] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:32 2022] DELETE /identity/v3/projects/32796d938707460a9a12104a204eacbe => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:51:32.920930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0cc2-3773-4814-a652-54f97b18759d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:32.921387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0cc2-3773-4814-a652-54f97b18759d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:32.921387 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0cc2-3773-4814-a652-54f97b18759d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:32.959876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bf2f0cc2-3773-4814-a652-54f97b18759d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:32.961499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bf2f0cc2-3773-4814-a652-54f97b18759d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:33.041128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1336/2725] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:33.049582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e95b-e931-45a9-9aa9-78cb77771cf2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.049888 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e95b-e931-45a9-9aa9-78cb77771cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.050175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e95b-e931-45a9-9aa9-78cb77771cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.078429 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4da1e95b-e931-45a9-9aa9-78cb77771cf2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:33.082873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4da1e95b-e931-45a9-9aa9-78cb77771cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:33.094887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1390/2726] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:33.099884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f0cc2-3773-4814-a652-54f97b18759d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:33.109932 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f0cc2-3773-4814-a652-54f97b18759d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:33.117466 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:33.118189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.118709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.119186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.120136 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:51:33.120636 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:33.122501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:33.141511 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1337/2727] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:51:33.143688 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1e95b-e931-45a9-9aa9-78cb77771cf2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:33.153763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1e95b-e931-45a9-9aa9-78cb77771cf2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:33.158894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:33.159512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.160903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.160903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.160903 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:33.163066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:33.300559 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1391/2728] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:51:33 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:51:33.303679 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:33.316213 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6f70e-9f68-4c9b-818c-307684891320 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:33.408915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:33.409569 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.409848 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.410161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.411503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:33.416166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:33.591067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1338/2729] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:51:33 2022] POST /identity/v3/users => generated 284 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:33.595513 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:33.606661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bc150-9a5b-4dff-a861-37ebf5997256 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:33.611942 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:33.612564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.612840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.613107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.613808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:33.615505 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:33.625956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1392/2730] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:33.630186 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:33.643854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c299f5-1c57-4f6f-b84e-a1ef754378b7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:33.648607 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88bee8-0fa4-4dc4-a325-40f4b5c514eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:33.649256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bee8-0fa4-4dc4-a325-40f4b5c514eb admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.649535 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bee8-0fa4-4dc4-a325-40f4b5c514eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.649807 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88bee8-0fa4-4dc4-a325-40f4b5c514eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.650658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bee8-0fa4-4dc4-a325-40f4b5c514eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:33.652251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88bee8-0fa4-4dc4-a325-40f4b5c514eb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:33.667336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1339/2731] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:33.668514 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:33.679415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd183d1-38a4-42b4-a62b-cf1198dcc73f admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:33.684104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:33.684894 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.685173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.685447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] PATH_INFO: `/v3/projects/445446ac509a46a3b069838fe929ab89/users/f2cc3deb0bbf4752b52c45e65414f4ec/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.701239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=445446ac509a46a3b069838fe929ab89, user_id=f2cc3deb0bbf4752b52c45e65414f4ec, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:33.703962 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:33.789381 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1393/2732] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:33 2022] PUT /identity/v3/projects/445446ac509a46a3b069838fe929ab89/users/f2cc3deb0bbf4752b52c45e65414f4ec/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:33.800065 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f72d7-507b-43e0-842c-fcee73ae9ad0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.800352 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f72d7-507b-43e0-842c-fcee73ae9ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.800701 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f72d7-507b-43e0-842c-fcee73ae9ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.827288 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:33.830710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-593f72d7-507b-43e0-842c-fcee73ae9ad0 None None] MFA Rules not processed for user `f2cc3deb0bbf4752b52c45e65414f4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:33.831927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-593f72d7-507b-43e0-842c-fcee73ae9ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:33.898356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1340/2733] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:33.902959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:33.967334 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c56d8-3bda-4eb5-8032-efc8dd99f6bc admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:33.973997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:33.974641 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:33.974918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:33.975188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:33.975852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:33.977790 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:34.101019 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1394/2734] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:51:33 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:34.105080 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f72d7-507b-43e0-842c-fcee73ae9ad0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:34.115820 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f72d7-507b-43e0-842c-fcee73ae9ad0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:34.170937 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:34.172630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:34.172630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:34.172630 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:34.172965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:34.175409 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:34.272064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1341/2735] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:51:34 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:34.276141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:34.291129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791aed0-4e74-4122-b03b-5af444649615 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:34.296155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:34.297346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:34.297947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:34.298989 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:34.302699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:34.302699 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:34.318415 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1395/2736] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:34.319706 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:34.332620 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729daa-5515-4e53-96c6-b5b535e631c4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:34.338431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:34.339281 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:34.339658 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:34.340033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] PATH_INFO: `/v3/projects/14e0b083d1ee4e9c8e5ddc41bf4cc5a2/users/0b844b335e294d41a1a05230dcde7429/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:34.357596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e0b083d1ee4e9c8e5ddc41bf4cc5a2, user_id=0b844b335e294d41a1a05230dcde7429, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:34.361959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:34.467247 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1342/2737] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:34 2022] PUT /identity/v3/projects/14e0b083d1ee4e9c8e5ddc41bf4cc5a2/users/0b844b335e294d41a1a05230dcde7429/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:34.472053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:34.495297 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3b20c-cf18-4685-baf4-aca0721e2103 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:34.541422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:34.558713 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c6737-fd48-4947-82a8-3c716fa87f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:34.559341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c6737-fd48-4947-82a8-3c716fa87f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:34.559620 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c6737-fd48-4947-82a8-3c716fa87f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:34.559893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c6737-fd48-4947-82a8-3c716fa87f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:34.560760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c6737-fd48-4947-82a8-3c716fa87f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:34.562457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c6737-fd48-4947-82a8-3c716fa87f81 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:34.574676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1396/2738] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:34.579033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:34.590866 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73025a0d-389a-47e1-9c65-27f5e1bf357b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:34.596207 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:34.597127 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:34.597472 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:34.597812 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] PATH_INFO: `/v3/projects/14e0b083d1ee4e9c8e5ddc41bf4cc5a2/users/0b844b335e294d41a1a05230dcde7429/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:34.599891 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e0b083d1ee4e9c8e5ddc41bf4cc5a2, user_id=0b844b335e294d41a1a05230dcde7429, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:34.602659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:34.613539 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1343/2739] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:34 2022] PUT /identity/v3/projects/14e0b083d1ee4e9c8e5ddc41bf4cc5a2/users/0b844b335e294d41a1a05230dcde7429/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:34.624638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:34.624956 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:34.625232 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:34.653564 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] MFA Rules not processed for user `0b844b335e294d41a1a05230dcde7429`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:34.654817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:34.725086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1397/2740] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:51:34.742333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:34.744901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:34.815276 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b509f1-9e38-452f-8133-e76f81d83495 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:34.819456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:34.820105 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:34.820386 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:34.820660 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:34.837102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:34.837102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:34.838323 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:34.847063 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1344/2741] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:34.986653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:34.988099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:35.000440 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b474d-9a7d-40b4-baf5-dbaca9e5eed4 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:35.004422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:35.005044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:35.005329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:35.005601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:35.070482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:35.072105 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:35.082898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1398/2742] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:37.163824 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:37.166267 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:37.235531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fb89d-827c-49af-8875-3757895f0d3f service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:37.240023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:37.240652 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:37.240929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:37.241200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] PATH_INFO: `/v3/users/7b63aaa3a9db4fa4b50bf93eae5a1ffc` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:37.242379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b63aaa3a9db4fa4b50bf93eae5a1ffc)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:37.244415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:37.555785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b63aaa3a9db4fa4b50bf93eae5a1ffc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:37.569075 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b63aaa3a9db4fa4b50bf93eae5a1ffc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:37.640659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b63aaa3a9db4fa4b50bf93eae5a1ffc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:37.649308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b63aaa3a9db4fa4b50bf93eae5a1ffc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:37.649971 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b63aaa3a9db4fa4b50bf93eae5a1ffc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:37.655958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1345/2743] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:37 2022] DELETE /identity/v3/users/7b63aaa3a9db4fa4b50bf93eae5a1ffc => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:37.913333 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:37.928633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6c981-f855-4ba4-a649-10d888e00b71 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:38.003231 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:38.004173 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:38.004655 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:38.004990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] PATH_INFO: `/v3/projects/5d10772d974f47d7bc82c0f0b8a8facf` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:38.006361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d10772d974f47d7bc82c0f0b8a8facf)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:38.008392 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:38.011267 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:38.180167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d10772d974f47d7bc82c0f0b8a8facf', 'request_id': 'req-57eaa059-5c26-4528-a030-9ad6148ed5bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.185517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d10772d974f47d7bc82c0f0b8a8facf', 'request_id': 'req-57eaa059-5c26-4528-a030-9ad6148ed5bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.299813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d10772d974f47d7bc82c0f0b8a8facf', 'request_id': 'req-57eaa059-5c26-4528-a030-9ad6148ed5bb'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.311774 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] The token cache is being invalidate because project 5d10772d974f47d7bc82c0f0b8a8facf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:38.312060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.312723 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.315371 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1399/2744] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:37 2022] DELETE /identity/v3/projects/5d10772d974f47d7bc82c0f0b8a8facf => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:51:38.319978 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:38.321509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:38.335315 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf123cb-7c56-447f-8999-1585ebdc95c2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:38.534521 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:38.544463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:38.544463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:38.544463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] PATH_INFO: `/v3/users/76eeb6a4cf0e44c299c476ad5e264a79` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:38.544463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] RBAC: Authorizing `identity:delete_user(user_id=76eeb6a4cf0e44c299c476ad5e264a79)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:38.544463 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:38.562499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:38.702752 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76eeb6a4cf0e44c299c476ad5e264a79'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.714968 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76eeb6a4cf0e44c299c476ad5e264a79'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.785618 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76eeb6a4cf0e44c299c476ad5e264a79'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.793399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76eeb6a4cf0e44c299c476ad5e264a79'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.794070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76eeb6a4cf0e44c299c476ad5e264a79'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:38.800337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1346/2745] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:38 2022] DELETE /identity/v3/users/76eeb6a4cf0e44c299c476ad5e264a79 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:39.078428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:39.080849 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:39.096211 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eaa059-5c26-4528-a030-9ad6148ed5bb admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:39.158203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:39.159184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:39.159485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:39.159789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] PATH_INFO: `/v3/projects/4a36d9652e80462a8e11c3b3e8a9c4a8` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:39.161042 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a36d9652e80462a8e11c3b3e8a9c4a8)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:39.163009 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:39.288123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a36d9652e80462a8e11c3b3e8a9c4a8', 'request_id': 'req-bd59b711-42c8-4027-afca-9f9b3c1db106'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:39.294093 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a36d9652e80462a8e11c3b3e8a9c4a8', 'request_id': 'req-bd59b711-42c8-4027-afca-9f9b3c1db106'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:39.367348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a36d9652e80462a8e11c3b3e8a9c4a8', 'request_id': 'req-bd59b711-42c8-4027-afca-9f9b3c1db106'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:39.380239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] The token cache is being invalidate because project 4a36d9652e80462a8e11c3b3e8a9c4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:39.380563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:39.381199 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-bd59b711-42c8-4027-afca-9f9b3c1db106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:39.382816 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1400/2746] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:39 2022] DELETE /identity/v3/projects/4a36d9652e80462a8e11c3b3e8a9c4a8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:40.665454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:40.666913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:40.679845 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00451d6b-d747-4b9e-a5fb-371f3eeb801a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:40.740012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:40.740012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:40.740012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:40.740012 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] PATH_INFO: `/v3/users/f2cc3deb0bbf4752b52c45e65414f4ec` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:40.740527 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2cc3deb0bbf4752b52c45e65414f4ec)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:40.741516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:40.923893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2cc3deb0bbf4752b52c45e65414f4ec'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:40.935841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2cc3deb0bbf4752b52c45e65414f4ec'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:40.984358 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2cc3deb0bbf4752b52c45e65414f4ec'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:40.992456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2cc3deb0bbf4752b52c45e65414f4ec'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:40.993118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2cc3deb0bbf4752b52c45e65414f4ec'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:41.003123 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1347/2747] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:40 2022] DELETE /identity/v3/users/f2cc3deb0bbf4752b52c45e65414f4ec => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:51:41.006536 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:41.007053 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:41.007166 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:41.026889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:41.028106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:41.045917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:41.088010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1401/2748] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:41.098846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:41.100469 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:41.119039 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:41.159710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc9de-39b4-4a5a-bdf2-ca36f559c1e5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:41.163245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:41.163854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:41.164130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:41.164400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:41.174640 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:41.176068 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:41.185751 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1348/2749] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:41.598449 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:41.599868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:41.612671 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617d7e1-93af-4da5-87ec-9a4d668d4e51 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:41.617411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:41.617913 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:41.618224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:41.618504 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] PATH_INFO: `/v3/projects/445446ac509a46a3b069838fe929ab89` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:41.619673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=445446ac509a46a3b069838fe929ab89)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:41.621590 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:41.728628 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445446ac509a46a3b069838fe929ab89', 'request_id': 'req-0accf640-0854-4df9-8855-7738215155f3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:41.734180 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445446ac509a46a3b069838fe929ab89', 'request_id': 'req-0accf640-0854-4df9-8855-7738215155f3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:41.790125 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445446ac509a46a3b069838fe929ab89', 'request_id': 'req-0accf640-0854-4df9-8855-7738215155f3'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:41.802045 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] The token cache is being invalidate because project 445446ac509a46a3b069838fe929ab89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:41.802366 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:41.803003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:41.804495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1402/2750] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:41 2022] DELETE /identity/v3/projects/445446ac509a46a3b069838fe929ab89 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:41.808531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:41.818644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3a688-d586-4be7-a6af-b865722dbb5c service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:41.873952 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:41.875363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:41.875363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:41.875363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] PATH_INFO: `/v3/users/0b844b335e294d41a1a05230dcde7429` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:41.876350 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b844b335e294d41a1a05230dcde7429)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:41.878392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:42.083760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b844b335e294d41a1a05230dcde7429'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.096515 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b844b335e294d41a1a05230dcde7429'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.173485 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b844b335e294d41a1a05230dcde7429'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.181238 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b844b335e294d41a1a05230dcde7429'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.181912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-35a53b0f-46d9-473a-ae29-df4e89165c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b844b335e294d41a1a05230dcde7429'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.191659 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1349/2751] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:41 2022] DELETE /identity/v3/users/0b844b335e294d41a1a05230dcde7429 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:42.602819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:42.604348 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:42.617580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accf640-0854-4df9-8855-7738215155f3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:42.677638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:42.678330 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:42.678603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:42.678873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] PATH_INFO: `/v3/projects/14e0b083d1ee4e9c8e5ddc41bf4cc5a2` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:42.680090 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] RBAC: Authorizing `identity:delete_project(project_id=14e0b083d1ee4e9c8e5ddc41bf4cc5a2)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:42.682079 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:42.806376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:42.807829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14e0b083d1ee4e9c8e5ddc41bf4cc5a2', 'request_id': 'req-b98c3c0a-ad69-41bd-8080-bb78584968db'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.813282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14e0b083d1ee4e9c8e5ddc41bf4cc5a2', 'request_id': 'req-b98c3c0a-ad69-41bd-8080-bb78584968db'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.886331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14e0b083d1ee4e9c8e5ddc41bf4cc5a2', 'request_id': 'req-b98c3c0a-ad69-41bd-8080-bb78584968db'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.899051 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] The token cache is being invalidate because project 14e0b083d1ee4e9c8e5ddc41bf4cc5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:51:42.899346 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.900003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-b98c3c0a-ad69-41bd-8080-bb78584968db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:42.901550 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1403/2752] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:51:42 2022] DELETE /identity/v3/projects/14e0b083d1ee4e9c8e5ddc41bf4cc5a2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:42.906454 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278b5de-48f5-4414-89f3-bcfbb511ba90 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:42.906945 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278b5de-48f5-4414-89f3-bcfbb511ba90 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:42.907043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278b5de-48f5-4414-89f3-bcfbb511ba90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:42.926690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6278b5de-48f5-4414-89f3-bcfbb511ba90 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:42.927895 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6278b5de-48f5-4414-89f3-bcfbb511ba90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:42.961882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:42.988192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1350/2753] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:43.055213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1744be-98ed-4461-b13f-5429a62acb54 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.055491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1744be-98ed-4461-b13f-5429a62acb54 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.055766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1744be-98ed-4461-b13f-5429a62acb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.075783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-3b1744be-98ed-4461-b13f-5429a62acb54 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:43.077007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3b1744be-98ed-4461-b13f-5429a62acb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:43.085907 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1404/2754] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:43.090085 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278b5de-48f5-4414-89f3-bcfbb511ba90 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.099983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278b5de-48f5-4414-89f3-bcfbb511ba90 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.104518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.105128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.105408 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.105680 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.106402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:51:43.106703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.108284 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:43.119896 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1351/2755] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:51:43.124213 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1744be-98ed-4461-b13f-5429a62acb54 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.134452 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1744be-98ed-4461-b13f-5429a62acb54 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.139106 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.139710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.139990 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.140264 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.140954 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.143891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:43.243770 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1405/2756] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:51:43 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:43.248091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.259097 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8cb60-fbec-4bd9-9bb6-23e27f63e6c9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.318449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.319180 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.319459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.319736 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.320397 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.322382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:43.385095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1352/2757] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:51:43 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:43.389537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.400251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f485-17ca-4cd9-b0a1-b611751c22be admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.404922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.405560 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.405829 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.406121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.406840 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.408653 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:43.419024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1406/2758] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:43.423672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.433543 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f935a-379d-43c8-b8ac-6d150f8c7cd4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.438007 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3dd99-6438-49c4-a749-1ccb429f8b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.439188 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3dd99-6438-49c4-a749-1ccb429f8b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.439460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3dd99-6438-49c4-a749-1ccb429f8b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.439721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3dd99-6438-49c4-a749-1ccb429f8b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.440402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3dd99-6438-49c4-a749-1ccb429f8b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.441969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3dd99-6438-49c4-a749-1ccb429f8b1b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:43.451854 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1353/2759] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:43.456150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.466372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69a979-c216-4810-8d75-967dc42f9fb8 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.471693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.472359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.472639 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.472912 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] PATH_INFO: `/v3/projects/fcf93fa0c2e5425bb87f1788cd3f200f/users/390b4810b7474988adf70b9b8bdc9e35/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.488634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf93fa0c2e5425bb87f1788cd3f200f, user_id=390b4810b7474988adf70b9b8bdc9e35, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.491565 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:43.555556 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1407/2760] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:43 2022] PUT /identity/v3/projects/fcf93fa0c2e5425bb87f1788cd3f200f/users/390b4810b7474988adf70b9b8bdc9e35/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:43.566645 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b2f1-7e4d-4ae4-ac1e-c1f050585589 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.566936 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b2f1-7e4d-4ae4-ac1e-c1f050585589 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.567204 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b2f1-7e4d-4ae4-ac1e-c1f050585589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.596060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d602b2f1-7e4d-4ae4-ac1e-c1f050585589 None None] MFA Rules not processed for user `390b4810b7474988adf70b9b8bdc9e35`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:43.597335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d602b2f1-7e4d-4ae4-ac1e-c1f050585589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:43.661927 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1354/2761] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:43.666443 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.728263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468081b7-8047-413c-9601-68517d4ccaff admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.732869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.733485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.733765 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.734037 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.734739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.737224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:43.850049 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1408/2762] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:51:43 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:43.854084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b2f1-7e4d-4ae4-ac1e-c1f050585589 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:43.865610 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b2f1-7e4d-4ae4-ac1e-c1f050585589 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:43.897227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:43.925519 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:43.926319 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:43.926480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:43.926987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:43.928236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:43.930230 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:44.047055 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1355/2763] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:51:43 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:51:44.053207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:44.063777 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c5650-2f03-45d9-9c2e-5d55d98f32e4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:44.069004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:44.069623 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:44.069904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:44.070218 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:44.070982 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:44.072578 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:44.085255 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1409/2764] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:51:44.089563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:44.101761 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08211bbd-8dd6-4d8b-8c1f-9366e8ce27a7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:44.106413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:44.107081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:44.107355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:44.107628 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] PATH_INFO: `/v3/projects/d998106ef4d940149bc476647543f13e/users/da458a9812124c11a88bdf1e29e7f4d0/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:44.122717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d998106ef4d940149bc476647543f13e, user_id=da458a9812124c11a88bdf1e29e7f4d0, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:44.125376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:44.252563 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1356/2765] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:44 2022] PUT /identity/v3/projects/d998106ef4d940149bc476647543f13e/users/da458a9812124c11a88bdf1e29e7f4d0/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:44.256574 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:44.267350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac205c5-5ce8-49b2-862d-a692d009f1e9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:44.312279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:44.325352 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb8d06-53da-4fbc-9575-f177290e9797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:44.325974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8d06-53da-4fbc-9575-f177290e9797 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:44.326295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8d06-53da-4fbc-9575-f177290e9797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:44.326573 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8d06-53da-4fbc-9575-f177290e9797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:44.327268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb8d06-53da-4fbc-9575-f177290e9797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:44.329050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb8d06-53da-4fbc-9575-f177290e9797 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:44.339734 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1410/2766] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:51:44.344181 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:44.354865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5ca55-d7a1-4343-b18a-495b1cb6f263 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:44.359587 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:44.360401 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:44.360532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:44.360792 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] PATH_INFO: `/v3/projects/d998106ef4d940149bc476647543f13e/users/da458a9812124c11a88bdf1e29e7f4d0/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:44.362786 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d998106ef4d940149bc476647543f13e, user_id=da458a9812124c11a88bdf1e29e7f4d0, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:44.365388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:44.376803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1357/2767] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:51:44 2022] PUT /identity/v3/projects/d998106ef4d940149bc476647543f13e/users/da458a9812124c11a88bdf1e29e7f4d0/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:44.387502 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:44.392307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:44.392307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:44.416423 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] MFA Rules not processed for user `da458a9812124c11a88bdf1e29e7f4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:44.417626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:44.491508 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1411/2768] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:44.502800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:44.504525 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:44.563900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e08fd9-a856-4440-846c-679850e0adac admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:44.567719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:44.568429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:44.568678 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:44.568953 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:44.626672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:44.628144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:44.637351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1358/2769] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:47.759486 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:47.760985 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:47.836086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53f789-6593-46f3-869a-0be036b92c60 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:47.841492 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:47.842107 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:47.842413 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:47.842685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:47.844889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:47.857470 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:47.859103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:47.871306 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1412/2770] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:50.051447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:50.060005 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a8ee5-9876-467a-bc6e-80fc0bdbea7e service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:50.064311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:50.065044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:50.065407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:50.065750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:50.075966 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:50.077577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:50.089412 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1359/2771] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:51.554924 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:51.556354 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:51.614733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2bd01-12b8-4c33-add0-f777cd5d7284 service nova] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:51.618434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe964834-a62d-459c-ac32-b17060b48ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:51.619038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964834-a62d-459c-ac32-b17060b48ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:51.619313 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964834-a62d-459c-ac32-b17060b48ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:51.619580 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964834-a62d-459c-ac32-b17060b48ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:51.621272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-fe964834-a62d-459c-ac32-b17060b48ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:51.628062 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:51.683703 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe964834-a62d-459c-ac32-b17060b48ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:51.685127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe964834-a62d-459c-ac32-b17060b48ca6 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:51.695250 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1413/2772] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:51:59.407585 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:59.437730 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:51:59.472973 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992a25d-a255-40f0-ac04-b2660bae5007 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:59.483772 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:59.484439 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:59.484719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:59.484990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] PATH_INFO: `/v3/users/390b4810b7474988adf70b9b8bdc9e35` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:59.486298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=390b4810b7474988adf70b9b8bdc9e35)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:59.489070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:59.695860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390b4810b7474988adf70b9b8bdc9e35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:59.713190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390b4810b7474988adf70b9b8bdc9e35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:59.795901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390b4810b7474988adf70b9b8bdc9e35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:59.804331 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390b4810b7474988adf70b9b8bdc9e35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:59.804983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390b4810b7474988adf70b9b8bdc9e35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:51:59.811122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1360/2773] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:51:59 2022] DELETE /identity/v3/users/390b4810b7474988adf70b9b8bdc9e35 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:51:59.816300 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:59.817060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:59.817060 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:59.836609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:51:59.837893 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:59.899305 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1414/2774] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:51:59.909125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:51:59.911322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:51:59.970164 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f07d1-55c4-4b87-8727-6b2f67eb1fe4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:51:59.974753 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:51:59.975393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:51:59.975673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:51:59.975947 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:51:59.985227 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:51:59.986732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:51:59.997212 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1361/2775] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:00.438331 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:00.440237 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:00.452908 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba91f8-7100-4eb5-97a0-0bf7228413ee None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:00.462947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:00.463603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:00.463925 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:00.464134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] PATH_INFO: `/v3/projects/fcf93fa0c2e5425bb87f1788cd3f200f` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:00.465310 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf93fa0c2e5425bb87f1788cd3f200f)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:00.467335 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:00.590004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf93fa0c2e5425bb87f1788cd3f200f', 'request_id': 'req-f830f204-ae0f-46cc-b01d-22fd90404182'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:00.595899 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf93fa0c2e5425bb87f1788cd3f200f', 'request_id': 'req-f830f204-ae0f-46cc-b01d-22fd90404182'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:00.670103 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf93fa0c2e5425bb87f1788cd3f200f', 'request_id': 'req-f830f204-ae0f-46cc-b01d-22fd90404182'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:00.685095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] The token cache is being invalidate because project fcf93fa0c2e5425bb87f1788cd3f200f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:00.685095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:00.685095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:00.685817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1415/2776] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:00 2022] DELETE /identity/v3/projects/fcf93fa0c2e5425bb87f1788cd3f200f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:00.689929 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:00.700611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c20c0f-9aa0-4f58-9cd7-bcbd7aff4878 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:00.748626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:00.760797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:00.761432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:00.761714 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:00.761989 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] PATH_INFO: `/v3/users/da458a9812124c11a88bdf1e29e7f4d0` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:00.763177 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da458a9812124c11a88bdf1e29e7f4d0)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:00.765116 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:00.947242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da458a9812124c11a88bdf1e29e7f4d0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:00.959548 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da458a9812124c11a88bdf1e29e7f4d0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.025337 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da458a9812124c11a88bdf1e29e7f4d0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.033342 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da458a9812124c11a88bdf1e29e7f4d0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.033986 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da458a9812124c11a88bdf1e29e7f4d0'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.040298 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1362/2777] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:52:00 2022] DELETE /identity/v3/users/da458a9812124c11a88bdf1e29e7f4d0 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:01.573647 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:01.574725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:01.587650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f204-ae0f-46cc-b01d-22fd90404182 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:01.615898 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:01.649372 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:01.650012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:01.650327 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:01.650601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] PATH_INFO: `/v3/projects/d998106ef4d940149bc476647543f13e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:01.651745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d998106ef4d940149bc476647543f13e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:01.653656 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:01.761251 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd998106ef4d940149bc476647543f13e', 'request_id': 'req-f69266f4-faaf-49f2-8f17-453318298a2d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.767028 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd998106ef4d940149bc476647543f13e', 'request_id': 'req-f69266f4-faaf-49f2-8f17-453318298a2d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.835460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:01.836894 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:01.840997 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd998106ef4d940149bc476647543f13e', 'request_id': 'req-f69266f4-faaf-49f2-8f17-453318298a2d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.851079 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f55cd-d94d-4444-816a-5714c1840a00 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:01.855043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] The token cache is being invalidate because project d998106ef4d940149bc476647543f13e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:01.856114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.856114 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-f69266f4-faaf-49f2-8f17-453318298a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:01.857689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1416/2778] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:01 2022] DELETE /identity/v3/projects/d998106ef4d940149bc476647543f13e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:01.863584 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fcedd-a3b0-4faa-993e-61f68192734d None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:01.864126 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fcedd-a3b0-4faa-993e-61f68192734d None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:01.864675 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fcedd-a3b0-4faa-993e-61f68192734d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:01.884397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-5a2fcedd-a3b0-4faa-993e-61f68192734d None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:01.885626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5a2fcedd-a3b0-4faa-993e-61f68192734d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:01.915320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:01.915969 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:01.916244 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:01.916508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] PATH_INFO: `/v3/users/8938189deb9549579aa69b7518e93edd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:01.917695 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8938189deb9549579aa69b7518e93edd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:01.919663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:01.948936 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1417/2779] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:01.957879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0522db0-af1c-4c5b-9756-b0f4ad1f86b0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:01.958164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0522db0-af1c-4c5b-9756-b0f4ad1f86b0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:01.958608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0522db0-af1c-4c5b-9756-b0f4ad1f86b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:01.978697 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-d0522db0-af1c-4c5b-9756-b0f4ad1f86b0 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:01.979890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d0522db0-af1c-4c5b-9756-b0f4ad1f86b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:01.987596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1418/2780] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:01.991755 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0522db0-af1c-4c5b-9756-b0f4ad1f86b0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.000800 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0522db0-af1c-4c5b-9756-b0f4ad1f86b0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.005402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.006099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.011013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.011013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.011013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:52:02.011013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.011013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.021676 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1419/2781] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 27 08:52:02.026083 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.036323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0e70f-9e72-4e19-ac2f-8b722aa156e0 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.040984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.041568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.041847 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.042119 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.042786 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.044659 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.101380 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8938189deb9549579aa69b7518e93edd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:02.113014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8938189deb9549579aa69b7518e93edd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:02.116951 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:02.123295 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1420/2782] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:52:02 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:52:02.127517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.146081 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:02.183924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8938189deb9549579aa69b7518e93edd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:02.189551 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2ee5c-87a2-4636-b628-8d11b8087699 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.191253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8938189deb9549579aa69b7518e93edd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:02.191920 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e1d9d3a-ef00-4a98-adee-e7dfc1f86455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8938189deb9549579aa69b7518e93edd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:02.194980 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.195654 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.195937 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.196215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.196910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.198873 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.199183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1363/2783] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:52:01 2022] DELETE /identity/v3/users/8938189deb9549579aa69b7518e93edd => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:52:02.203549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.203835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.204115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.225724 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:02.225956 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:02.284192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1364/2784] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:02.294441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.295780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:02.303155 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1421/2785] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:52:02 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:52:02.304302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.306121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:02.318627 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411334b7-7ad1-4ee0-a83a-40536547e92e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.323294 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.323909 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.324188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.324464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.325160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.326923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.340350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1422/2786] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:02.341544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.352332 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1314b3-bb7c-4f21-8331-96bc81152b46 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.354558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7453675-118d-4f2f-a900-e61cc44dc263 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.356896 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.361979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.361979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.361979 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.362294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c8b29-1c9a-4093-9f67-0793256443ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.362421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.362547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c8b29-1c9a-4093-9f67-0793256443ed service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.362547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c8b29-1c9a-4093-9f67-0793256443ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.362547 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c8b29-1c9a-4093-9f67-0793256443ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.362796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.369205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c8b29-1c9a-4093-9f67-0793256443ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.370793 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c8b29-1c9a-4093-9f67-0793256443ed service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.373856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1423/2787] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:52:02.377974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.380979 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1365/2788] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:52:02.388433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc3de3-8875-4ba1-9f26-2d96d62ef032 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.393157 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.393817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.394097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.394402 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] PATH_INFO: `/v3/projects/a257f473a81b4e76a9ce7117940fcf9e/users/9435930bddd54587a5f72d98002d5d48/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.409323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a257f473a81b4e76a9ce7117940fcf9e, user_id=9435930bddd54587a5f72d98002d5d48, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.412891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.509172 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1424/2789] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:02 2022] PUT /identity/v3/projects/a257f473a81b4e76a9ce7117940fcf9e/users/9435930bddd54587a5f72d98002d5d48/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:52:02.520869 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.521167 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.521431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.560431 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] MFA Rules not processed for user `9435930bddd54587a5f72d98002d5d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:02.561684 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:02.635993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1366/2790] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:02.648323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.650133 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:02.710271 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f66f86-2dd3-4bb1-8f5c-a14b0895d837 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.713877 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625c7bb-9d28-48bf-92b8-8008740435bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.714520 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625c7bb-9d28-48bf-92b8-8008740435bf service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.714797 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625c7bb-9d28-48bf-92b8-8008740435bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.715066 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625c7bb-9d28-48bf-92b8-8008740435bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.725831 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625c7bb-9d28-48bf-92b8-8008740435bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.726014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625c7bb-9d28-48bf-92b8-8008740435bf service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.735447 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1425/2791] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:02.775317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:02.777718 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:02.816028 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:02.866101 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc4a6f-b5c6-457a-b68a-ef32b4fde626 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:02.870710 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:02.871415 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:02.871700 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:02.871970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] PATH_INFO: `/v3/projects/0415bf840a6f452d995aecae583eaa07` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:02.873131 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0415bf840a6f452d995aecae583eaa07)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:02.875049 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:02.990165 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0415bf840a6f452d995aecae583eaa07', 'request_id': 'req-a35632fa-838f-4506-b380-87c41b1a4a8f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:02.996138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0415bf840a6f452d995aecae583eaa07', 'request_id': 'req-a35632fa-838f-4506-b380-87c41b1a4a8f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:03.002970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0415bf840a6f452d995aecae583eaa07', 'request_id': 'req-a35632fa-838f-4506-b380-87c41b1a4a8f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:03.092064 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] The token cache is being invalidate because project 0415bf840a6f452d995aecae583eaa07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:03.092351 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:03.093219 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a35632fa-838f-4506-b380-87c41b1a4a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:03.094901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1367/2792] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:02 2022] DELETE /identity/v3/projects/0415bf840a6f452d995aecae583eaa07 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:03.099883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937bd8-52cb-4cbc-9d53-c929592e300f None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.100170 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937bd8-52cb-4cbc-9d53-c929592e300f None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.100454 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937bd8-52cb-4cbc-9d53-c929592e300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.120863 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-c3937bd8-52cb-4cbc-9d53-c929592e300f None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:03.122097 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c3937bd8-52cb-4cbc-9d53-c929592e300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:03.184848 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1426/2793] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:03.193214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a00e-d163-43d4-8712-afca66e02c35 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.193498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a00e-d163-43d4-8712-afca66e02c35 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.193773 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a00e-d163-43d4-8712-afca66e02c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.214443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-50a4a00e-d163-43d4-8712-afca66e02c35 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:03.215648 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-50a4a00e-d163-43d4-8712-afca66e02c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:03.224202 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1368/2794] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:03.228476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937bd8-52cb-4cbc-9d53-c929592e300f None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:03.233427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:03.238750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937bd8-52cb-4cbc-9d53-c929592e300f None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:03.243720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:03.244343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.244544 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.244814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.245534 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:52:03.245834 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:03.247457 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:03.259754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1427/2795] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:52:03.264265 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4a00e-d163-43d4-8712-afca66e02c35 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:03.274357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4a00e-d163-43d4-8712-afca66e02c35 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:03.278994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:03.279600 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.279872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.280144 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.280802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:03.282727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:03.402788 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1369/2796] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:52:03 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:52:03.407010 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:03.417545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144ef66-7bcc-4e9a-a6a5-847458cd077e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:03.474329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:03.474959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.475235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.475503 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.476179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:03.478100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:03.581679 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1428/2797] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:52:03 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:03.581825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:03.589441 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b564aef-c0bd-4ed4-8184-7f3606b9f102 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:03.594419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:03.595048 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.595322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.595591 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.596275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:03.597846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:03.608259 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1370/2798] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:03.616914 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:03.627375 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8abb-1469-4766-8802-6be589882529 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:03.633043 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0039e45-eaf0-4fb4-b90b-3c48d678214d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:03.633650 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039e45-eaf0-4fb4-b90b-3c48d678214d admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.633987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039e45-eaf0-4fb4-b90b-3c48d678214d admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.634312 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0039e45-eaf0-4fb4-b90b-3c48d678214d admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.635006 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0039e45-eaf0-4fb4-b90b-3c48d678214d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:03.636638 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0039e45-eaf0-4fb4-b90b-3c48d678214d admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:03.647820 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1429/2799] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 27 08:52:03.655205 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:03.665091 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005d5e1-cae9-4695-aa9f-2a063e8abb8a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:03.669901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:03.670627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.670907 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.671236 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] PATH_INFO: `/v3/projects/8fa646984231427a93274cd145d0fdc0/users/d0f3fb978b0340c6b888dca6b1fd95c6/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.689512 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa646984231427a93274cd145d0fdc0, user_id=d0f3fb978b0340c6b888dca6b1fd95c6, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:03.692841 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:03.791019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1371/2800] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:03 2022] PUT /identity/v3/projects/8fa646984231427a93274cd145d0fdc0/users/d0f3fb978b0340c6b888dca6b1fd95c6/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:03.802125 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1887cf-31b5-4038-90b5-395ade4be6cb None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.802633 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1887cf-31b5-4038-90b5-395ade4be6cb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.802736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1887cf-31b5-4038-90b5-395ade4be6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.830215 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-8b1887cf-31b5-4038-90b5-395ade4be6cb None None] MFA Rules not processed for user `d0f3fb978b0340c6b888dca6b1fd95c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:03.831435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8b1887cf-31b5-4038-90b5-395ade4be6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:03.896038 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1430/2801] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:03.900556 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:03.950077 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:03.963858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed50fae-7c23-449f-b377-0619ed67d03d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:03.968734 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:03.969341 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:03.969616 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:03.969885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:03.970589 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:03.972522 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:04.070959 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1372/2802] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:52:03 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:04.075034 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1887cf-31b5-4038-90b5-395ade4be6cb None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:04.086236 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1887cf-31b5-4038-90b5-395ade4be6cb None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:04.138693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:04.144901 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:04.146115 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.146416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.146714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.147412 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:04.149359 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:04.264568 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1431/2803] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:52:04 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:04.268609 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:04.279453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77cd82c-1c00-4bd5-b7a6-0bec3a468ceb admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:04.283961 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:04.284562 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.284842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.285139 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.285838 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:04.288239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:04.298122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1373/2804] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:04.302007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:04.312483 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6d93b-0f79-4765-9bc6-30bb3656ee8e admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:04.317347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676f932-ba69-4deb-bf35-cfb2f381d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:04.318189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676f932-ba69-4deb-bf35-cfb2f381d5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.318588 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676f932-ba69-4deb-bf35-cfb2f381d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.318872 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676f932-ba69-4deb-bf35-cfb2f381d5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.319618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676f932-ba69-4deb-bf35-cfb2f381d5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:04.325425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676f932-ba69-4deb-bf35-cfb2f381d5cf admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:04.331453 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1432/2805] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:04.336427 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:04.347200 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc39980-7b11-4b22-9e00-f802c2e4b2e9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:04.352518 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:04.353173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.359908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.359908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] PATH_INFO: `/v3/projects/3a29f71a5e5548719cc603709bd45f20/users/6e5c3280e7484fe7b43a8ee3d95c21c3/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.368391 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a29f71a5e5548719cc603709bd45f20, user_id=6e5c3280e7484fe7b43a8ee3d95c21c3, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:04.371019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:04.465861 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1374/2806] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:04 2022] PUT /identity/v3/projects/3a29f71a5e5548719cc603709bd45f20/users/6e5c3280e7484fe7b43a8ee3d95c21c3/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:04.535376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb740-ff14-4984-9f24-c1ff5c518ede None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.535660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb740-ff14-4984-9f24-c1ff5c518ede None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.535935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb740-ff14-4984-9f24-c1ff5c518ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.567658 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-cb7fb740-ff14-4984-9f24-c1ff5c518ede None None] MFA Rules not processed for user `6e5c3280e7484fe7b43a8ee3d95c21c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:04.568509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cb7fb740-ff14-4984-9f24-c1ff5c518ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:04.637222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1433/2807] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:04.641871 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:04.703832 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe48338-c27c-4126-ae3b-e0b8846f7fab admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:04.709376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:04.709997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.710305 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.710578 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.711240 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:04.713122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:04.804031 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1375/2808] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:52:04 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:04.808329 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fb740-ff14-4984-9f24-c1ff5c518ede None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:04.819838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fb740-ff14-4984-9f24-c1ff5c518ede None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:04.878323 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:04.878950 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.879224 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.879493 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.880150 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:04.882092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:04.967347 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1434/2809] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:52:04 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:04.971603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:04.982796 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155337f1-9458-4bf0-a67b-a5ba2a7209b0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:04.987689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:04.988300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:04.988574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:04.996491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:04.996491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:04.996491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:05.002248 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1376/2810] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:05.006875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:05.017315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53edc2-b1bc-463b-97ca-c2c3d86e72b9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:05.022667 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:05.023342 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:05.023616 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:05.023889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] PATH_INFO: `/v3/projects/7d8594a72e3341349fc32c2e49919e84/users/4cabb47dacfe414bb11c1a9ede6b2e0e/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:05.040981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8594a72e3341349fc32c2e49919e84, user_id=4cabb47dacfe414bb11c1a9ede6b2e0e, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:05.043677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:05.104789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1435/2811] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:05 2022] PUT /identity/v3/projects/7d8594a72e3341349fc32c2e49919e84/users/4cabb47dacfe414bb11c1a9ede6b2e0e/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:05.108860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:05.119803 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3335-b24d-4851-a073-988f95f9736d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:05.179295 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2f1d5-eb48-49f5-97ab-aa6de65627f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:05.179916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f1d5-eb48-49f5-97ab-aa6de65627f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:05.180195 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f1d5-eb48-49f5-97ab-aa6de65627f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:05.180489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2f1d5-eb48-49f5-97ab-aa6de65627f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:05.181162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2f1d5-eb48-49f5-97ab-aa6de65627f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:05.182872 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2f1d5-eb48-49f5-97ab-aa6de65627f3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:05.193662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1377/2812] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:05.197796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:05.208396 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d76ca-5227-4e3c-972c-2b914b471425 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:05.217421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:05.217421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:05.217421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:05.217421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] PATH_INFO: `/v3/projects/7d8594a72e3341349fc32c2e49919e84/users/4cabb47dacfe414bb11c1a9ede6b2e0e/roles/3cf29058f0f54877aa0a7863c00bc628` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:05.220690 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8594a72e3341349fc32c2e49919e84, user_id=4cabb47dacfe414bb11c1a9ede6b2e0e, role_id=3cf29058f0f54877aa0a7863c00bc628)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:05.224050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:05.236637 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1436/2813] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:05 2022] PUT /identity/v3/projects/7d8594a72e3341349fc32c2e49919e84/users/4cabb47dacfe414bb11c1a9ede6b2e0e/roles/3cf29058f0f54877aa0a7863c00bc628 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:52:05.247089 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:05.247382 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:05.247663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:05.262995 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:05.277851 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] MFA Rules not processed for user `4cabb47dacfe414bb11c1a9ede6b2e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:05.279117 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:05.353086 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1378/2814] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:05.364455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:05.366410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:05.388390 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:05.427142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092d4c8-bdf2-454c-8b0f-2861a04f5cf3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:05.430926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c18d46-7a86-4ffb-adb4-72bab6abc591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:05.431561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c18d46-7a86-4ffb-adb4-72bab6abc591 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:05.431841 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c18d46-7a86-4ffb-adb4-72bab6abc591 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:05.432116 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c18d46-7a86-4ffb-adb4-72bab6abc591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:05.499691 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c18d46-7a86-4ffb-adb4-72bab6abc591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:05.501162 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c18d46-7a86-4ffb-adb4-72bab6abc591 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:05.510760 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1437/2815] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:06.855035 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:06.861084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:06.932540 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67536ca-9300-416f-a7c8-bc66e76e0c25 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:06.937095 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:06.937777 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:06.938046 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:06.938347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] PATH_INFO: `/v3/users/9435930bddd54587a5f72d98002d5d48` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:06.939544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9435930bddd54587a5f72d98002d5d48)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:06.941491 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:07.158435 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9435930bddd54587a5f72d98002d5d48'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:07.174137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9435930bddd54587a5f72d98002d5d48'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:07.292405 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9435930bddd54587a5f72d98002d5d48'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:07.305613 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9435930bddd54587a5f72d98002d5d48'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:07.306672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9435930bddd54587a5f72d98002d5d48'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:07.314993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1379/2816] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:52:06 2022] DELETE /identity/v3/users/9435930bddd54587a5f72d98002d5d48 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:07.320253 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:07.320904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:07.320904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:07.341024 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:07.342316 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:07.415277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1438/2817] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:07.425162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:07.426603 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:07.444750 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:07.494859 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033cc54-2faf-40b1-9cae-6ed189991e2b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:07.498617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:07.499239 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:07.499513 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:07.499784 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:07.512318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:07.514314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:07.529018 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1380/2818] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:07.679833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:07.679833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:07.743408 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f393c-0646-4686-b99a-d20530693fb2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:07.747283 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9e14c-af6a-4862-bc68-9715618434ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:07.747926 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9e14c-af6a-4862-bc68-9715618434ad service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:07.748206 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9e14c-af6a-4862-bc68-9715618434ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:07.748475 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9e14c-af6a-4862-bc68-9715618434ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:07.750424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f7b9e14c-af6a-4862-bc68-9715618434ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:07.784109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:07.826124 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9e14c-af6a-4862-bc68-9715618434ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:07.827626 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9e14c-af6a-4862-bc68-9715618434ad service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:07.839813 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1439/2819] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:08.342524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:08.347185 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:08.371394 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358704be-27bf-48eb-ab9c-434ea5f31ebf service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:08.379392 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:08.380314 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:08.380644 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:08.380928 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] PATH_INFO: `/v3/projects/a257f473a81b4e76a9ce7117940fcf9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:08.383078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a257f473a81b4e76a9ce7117940fcf9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:08.389329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:08.644722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a257f473a81b4e76a9ce7117940fcf9e', 'request_id': 'req-c22e2857-e353-41f1-8c3d-1f9901007cf2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:08.650432 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a257f473a81b4e76a9ce7117940fcf9e', 'request_id': 'req-c22e2857-e353-41f1-8c3d-1f9901007cf2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:08.656672 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a257f473a81b4e76a9ce7117940fcf9e', 'request_id': 'req-c22e2857-e353-41f1-8c3d-1f9901007cf2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:08.848482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] The token cache is being invalidate because project a257f473a81b4e76a9ce7117940fcf9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:08.848760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:08.849390 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c22e2857-e353-41f1-8c3d-1f9901007cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:08.851043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1381/2820] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:08 2022] DELETE /identity/v3/projects/a257f473a81b4e76a9ce7117940fcf9e => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:08.856889 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241c4e6-660c-47e2-b715-912ef7159039 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:08.857769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241c4e6-660c-47e2-b715-912ef7159039 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:08.857769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241c4e6-660c-47e2-b715-912ef7159039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:08.880567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-e241c4e6-660c-47e2-b715-912ef7159039 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:08.882768 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e241c4e6-660c-47e2-b715-912ef7159039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:08.950766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1440/2821] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:08.959882 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba46fc-c250-45d5-8738-acaa5d77dbb7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:08.960179 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba46fc-c250-45d5-8738-acaa5d77dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:08.960997 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba46fc-c250-45d5-8738-acaa5d77dbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:08.983760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eeba46fc-c250-45d5-8738-acaa5d77dbb7 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:08.986275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eeba46fc-c250-45d5-8738-acaa5d77dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:08.992994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1382/2822] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:08.998189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241c4e6-660c-47e2-b715-912ef7159039 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:09.015902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241c4e6-660c-47e2-b715-912ef7159039 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:09.024374 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:09.025427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:09.025932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:09.026464 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:09.032538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:52:09.032538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:09.032538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:09.052669 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1441/2823] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:52:09.057732 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba46fc-c250-45d5-8738-acaa5d77dbb7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:09.068115 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba46fc-c250-45d5-8738-acaa5d77dbb7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:09.072990 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:09.073697 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:09.073985 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:09.074294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:09.074984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:09.077138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:09.192210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1383/2824] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:52:09 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:52:09.196466 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:09.206827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9400e-18e2-448f-b73a-8a4dc8ed90f4 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:09.275252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:09.275878 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:09.276154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:09.276422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:09.277129 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:09.279118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:09.435121 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1442/2825] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:52:09 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 27 08:52:09.439393 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:09.450996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb603c93-c68b-403e-b7a6-60c3350c34a7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:09.455564 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:09.456173 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:09.456449 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:09.456715 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:09.457402 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:09.459043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:09.470355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1384/2826] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:09.492710 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:09.504521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b867-4e93-475b-b929-2990e055e2a9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:09.510416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2762b6-d3d7-42fd-aa1d-903547cf780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:09.511007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2762b6-d3d7-42fd-aa1d-903547cf780c admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:09.511325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2762b6-d3d7-42fd-aa1d-903547cf780c admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:09.511600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2762b6-d3d7-42fd-aa1d-903547cf780c admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:09.512561 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2762b6-d3d7-42fd-aa1d-903547cf780c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:09.514132 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2762b6-d3d7-42fd-aa1d-903547cf780c admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:09.528794 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1443/2827] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:09.533148 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:09.543245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df96283-fbac-4307-859d-a9af0558588e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:09.553572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:09.553572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:09.553572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:09.553572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] PATH_INFO: `/v3/projects/23575e72cc5443f682b5387dd62305a5/users/edb4ee9715d946e6bd520cb5622162b7/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:09.564983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23575e72cc5443f682b5387dd62305a5, user_id=edb4ee9715d946e6bd520cb5622162b7, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:09.567674 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:09.773762 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1385/2828] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:09 2022] PUT /identity/v3/projects/23575e72cc5443f682b5387dd62305a5/users/edb4ee9715d946e6bd520cb5622162b7/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:52:09.792733 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:09.793017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:09.793299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:09.823672 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] MFA Rules not processed for user `edb4ee9715d946e6bd520cb5622162b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:09.824902 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:09.846932 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:09.891845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1444/2829] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:09.907687 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:09.909456 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:09.955489 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:09.996437 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d6a6-7c5c-440a-94a3-83ce0e0dddd2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:10.001800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e41212825ad4990958a7e909c669177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:10.001800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:10.002041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:10.002336 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:10.011717 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:10.013214 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:10.022493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1386/2830] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:52:10.174069 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:10.175678 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:10.239325 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a04b83-b40d-4c3c-9c67-886b3181e448 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:10.243629 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c308139-e592-4cb3-822d-9929989d41f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:10.244245 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c308139-e592-4cb3-822d-9929989d41f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:10.244554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c308139-e592-4cb3-822d-9929989d41f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:10.244824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c308139-e592-4cb3-822d-9929989d41f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:10.246832 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-8c308139-e592-4cb3-822d-9929989d41f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:10.318696 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c308139-e592-4cb3-822d-9929989d41f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:10.320766 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c308139-e592-4cb3-822d-9929989d41f2 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:10.331080 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1445/2831] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:15.267901 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:15.269338 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:15.345690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737e2bd-3f53-4ae7-82a3-77db0a60df56 service glance] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:15.350663 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:15.351345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:15.351746 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:15.352032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] PATH_INFO: `/v3/users/d0f3fb978b0340c6b888dca6b1fd95c6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:15.353261 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f3fb978b0340c6b888dca6b1fd95c6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:15.355965 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:15.655450 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f3fb978b0340c6b888dca6b1fd95c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:15.668908 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f3fb978b0340c6b888dca6b1fd95c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:15.743727 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f3fb978b0340c6b888dca6b1fd95c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:15.751722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f3fb978b0340c6b888dca6b1fd95c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:15.752419 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f3fb978b0340c6b888dca6b1fd95c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:15.758532 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1387/2832] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:52:15 2022] DELETE /identity/v3/users/d0f3fb978b0340c6b888dca6b1fd95c6 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:15.763674 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:15.764252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:15.764252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:15.784725 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:15.785945 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:15.852201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1446/2833] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:15.863795 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:15.868498 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:15.914780 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:15.934184 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e2696-9581-4f67-bea5-45f5b88b985e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:15.939156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:15.939156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:15.939156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:15.939156 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:15.951043 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:15.952524 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:15.963335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1388/2834] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:16.448190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:16.448190 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:16.457460 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cf7f6-08b0-4996-9a6b-de55fbf0e2d2 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:16.462343 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:16.463059 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:16.463361 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:16.463661 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] PATH_INFO: `/v3/projects/8fa646984231427a93274cd145d0fdc0` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:16.465814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa646984231427a93274cd145d0fdc0)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:16.467808 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:16.481527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:16.624393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa646984231427a93274cd145d0fdc0', 'request_id': 'req-e7755502-5955-41fb-9911-d9896bf50e4a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:16.630798 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa646984231427a93274cd145d0fdc0', 'request_id': 'req-e7755502-5955-41fb-9911-d9896bf50e4a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:16.738282 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa646984231427a93274cd145d0fdc0', 'request_id': 'req-e7755502-5955-41fb-9911-d9896bf50e4a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:16.752411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] The token cache is being invalidate because project 8fa646984231427a93274cd145d0fdc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:16.757852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:16.757852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:16.757852 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1447/2835] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:16 2022] DELETE /identity/v3/projects/8fa646984231427a93274cd145d0fdc0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:16.758996 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:16.769888 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b08989-ff74-4256-9d8a-b851cef3891d service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:16.831835 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:16.832495 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:16.833196 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:16.833473 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] PATH_INFO: `/v3/users/6e5c3280e7484fe7b43a8ee3d95c21c3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:16.834733 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e5c3280e7484fe7b43a8ee3d95c21c3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:16.836756 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:17.063246 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e5c3280e7484fe7b43a8ee3d95c21c3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:17.076509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e5c3280e7484fe7b43a8ee3d95c21c3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:17.161426 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e5c3280e7484fe7b43a8ee3d95c21c3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:17.171520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e5c3280e7484fe7b43a8ee3d95c21c3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:17.172406 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e5c3280e7484fe7b43a8ee3d95c21c3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:17.181294 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1389/2836] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:52:16 2022] DELETE /identity/v3/users/6e5c3280e7484fe7b43a8ee3d95c21c3 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:17.683916 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:17.686737 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:17.699600 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7755502-5955-41fb-9911-d9896bf50e4a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:17.764811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:17.765582 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:17.765861 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:17.766136 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] PATH_INFO: `/v3/projects/3a29f71a5e5548719cc603709bd45f20` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:17.772526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a29f71a5e5548719cc603709bd45f20)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:17.772526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:17.957479 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a29f71a5e5548719cc603709bd45f20', 'request_id': 'req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:17.964618 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a29f71a5e5548719cc603709bd45f20', 'request_id': 'req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.086917 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a29f71a5e5548719cc603709bd45f20', 'request_id': 'req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.101207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] The token cache is being invalidate because project 3a29f71a5e5548719cc603709bd45f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:18.101495 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.102217 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.103796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1448/2837] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:17 2022] DELETE /identity/v3/projects/3a29f71a5e5548719cc603709bd45f20 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:18.109189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:18.110775 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:18.124515 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ed55-5636-4086-a1ea-3375ec253a5d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:18.179166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:18.186842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:18.187515 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:18.187802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:18.188084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] PATH_INFO: `/v3/users/4cabb47dacfe414bb11c1a9ede6b2e0e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:18.189611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cabb47dacfe414bb11c1a9ede6b2e0e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:18.191599 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:18.610138 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cabb47dacfe414bb11c1a9ede6b2e0e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.622633 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cabb47dacfe414bb11c1a9ede6b2e0e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.681242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cabb47dacfe414bb11c1a9ede6b2e0e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.689709 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cabb47dacfe414bb11c1a9ede6b2e0e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.690501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-778425ca-31fd-47f9-a142-d4c22ca0b98e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cabb47dacfe414bb11c1a9ede6b2e0e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:18.697916 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1390/2838] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:52:18 2022] DELETE /identity/v3/users/4cabb47dacfe414bb11c1a9ede6b2e0e => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:19.175921 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:19.177382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:19.193731 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f0eee-2ac6-46c1-aa8b-89cba387ed52 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:19.225416 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:19.265595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:19.266277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:19.266554 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:19.266824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] PATH_INFO: `/v3/projects/7d8594a72e3341349fc32c2e49919e84` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:19.268014 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d8594a72e3341349fc32c2e49919e84)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:19.270000 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:19.384220 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d8594a72e3341349fc32c2e49919e84', 'request_id': 'req-96df864c-dce8-459a-941d-0f0ac89a6b69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:19.393328 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d8594a72e3341349fc32c2e49919e84', 'request_id': 'req-96df864c-dce8-459a-941d-0f0ac89a6b69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:19.515720 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d8594a72e3341349fc32c2e49919e84', 'request_id': 'req-96df864c-dce8-459a-941d-0f0ac89a6b69'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:19.528287 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] The token cache is being invalidate because project 7d8594a72e3341349fc32c2e49919e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:19.528577 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:19.529563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-96df864c-dce8-459a-941d-0f0ac89a6b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:19.531233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1449/2839] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:19 2022] DELETE /identity/v3/projects/7d8594a72e3341349fc32c2e49919e84 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:19.537137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90d442-281c-44c3-9f2d-c21e0736859b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:19.537638 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90d442-281c-44c3-9f2d-c21e0736859b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:19.541416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90d442-281c-44c3-9f2d-c21e0736859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:19.559883 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4b90d442-281c-44c3-9f2d-c21e0736859b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:19.561130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4b90d442-281c-44c3-9f2d-c21e0736859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:19.628673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1391/2840] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:19.636944 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273e1d-e7ff-4064-83c0-e8f177200873 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:19.637233 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273e1d-e7ff-4064-83c0-e8f177200873 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:19.637537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4273e1d-e7ff-4064-83c0-e8f177200873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:19.659160 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-f4273e1d-e7ff-4064-83c0-e8f177200873 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:19.660370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f4273e1d-e7ff-4064-83c0-e8f177200873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:19.670419 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1450/2841] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:19.674924 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90d442-281c-44c3-9f2d-c21e0736859b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:19.684767 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90d442-281c-44c3-9f2d-c21e0736859b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:19.690283 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:19.690899 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:19.691178 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:19.691442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:19.692245 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:52:19.692580 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:19.694451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:19.708270 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1392/2842] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:52:19.712819 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4273e1d-e7ff-4064-83c0-e8f177200873 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:19.723586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4273e1d-e7ff-4064-83c0-e8f177200873 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:19.728296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:19.728911 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:19.729189 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:19.729459 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:19.730161 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:19.732241 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:19.837400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1451/2843] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:52:19 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:19.844739 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:19.854029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17009282-2773-4076-bb06-0a657af439d2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:19.916311 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:19.916982 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:19.917328 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:19.917682 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:19.918396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:19.920356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:19.930785 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:20.087395 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1393/2844] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:52:19 2022] POST /identity/v3/users => generated 288 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:20.092785 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:20.103793 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e2991-2335-46cb-b32e-587e68548c86 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:20.109175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:20.109811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:20.110239 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:20.110521 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:20.111261 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:20.113164 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:20.124535 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1452/2845] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:20.131823 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:20.143617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371dfd0-ae0e-4b40-bb10-cbaf7ada3774 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:20.149057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb1598-bfa6-4bfb-ae5e-f47ca89a3118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:20.149714 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb1598-bfa6-4bfb-ae5e-f47ca89a3118 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:20.150033 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb1598-bfa6-4bfb-ae5e-f47ca89a3118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:20.150383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb1598-bfa6-4bfb-ae5e-f47ca89a3118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:20.151125 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb1598-bfa6-4bfb-ae5e-f47ca89a3118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:20.152865 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb1598-bfa6-4bfb-ae5e-f47ca89a3118 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:20.165376 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1394/2846] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:20.170341 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:20.180743 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887db78-094b-4a4e-8554-4c48fc140c8a admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:20.185884 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:20.186608 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:20.186885 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:20.187200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] PATH_INFO: `/v3/projects/23200086cd3246318074e7eafa118e58/users/c9a0f644e9794e51b843952b3552cd22/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:20.204787 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23200086cd3246318074e7eafa118e58, user_id=c9a0f644e9794e51b843952b3552cd22, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:20.207526 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:20.339234 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1453/2847] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:20 2022] PUT /identity/v3/projects/23200086cd3246318074e7eafa118e58/users/c9a0f644e9794e51b843952b3552cd22/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:20.351596 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44b97-0c60-47cc-99a4-59fbddefc761 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:20.351802 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44b97-0c60-47cc-99a4-59fbddefc761 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:20.352057 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44b97-0c60-47cc-99a4-59fbddefc761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:20.393304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-37a44b97-0c60-47cc-99a4-59fbddefc761 None None] MFA Rules not processed for user `c9a0f644e9794e51b843952b3552cd22`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:20.393304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-37a44b97-0c60-47cc-99a4-59fbddefc761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:20.462509 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1395/2848] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:20.474673 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:20.476845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:20.555404 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440d25-4d84-46bf-96be-6b070c220ef9 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:20.560364 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:20.560984 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:20.561263 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:20.561583 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:20.569537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:20.573981 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:20.575575 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:20.585501 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1454/2849] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:23.602322 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abda5a-df88-4488-a1fc-6bdc6af9d248 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:23.602860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abda5a-df88-4488-a1fc-6bdc6af9d248 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:23.602860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abda5a-df88-4488-a1fc-6bdc6af9d248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:23.621980 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-62abda5a-df88-4488-a1fc-6bdc6af9d248 None None] MFA Rules not processed for user `c9a0f644e9794e51b843952b3552cd22`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:23.623806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-62abda5a-df88-4488-a1fc-6bdc6af9d248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:23.631656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1396/2850] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:30.703700 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=2 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:33.616502 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=1,keystone:SELECT=42 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:47.890711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:47.892200 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:47.977317 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137c422-8f61-4cb7-8576-324baf38cc20 service cinder] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:47.982424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:47.983044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:47.983326 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:47.983595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:47.985356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:47.997524 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:47.999127 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:48.009868 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1455/2851] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:49.958903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1cb5-70e1-4b87-b144-7be3c42744de None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:49.958903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1cb5-70e1-4b87-b144-7be3c42744de None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:49.958903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1cb5-70e1-4b87-b144-7be3c42744de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:50.010665 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-99eb1cb5-70e1-4b87-b144-7be3c42744de None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:50.012508 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-99eb1cb5-70e1-4b87-b144-7be3c42744de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:50.087100 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1397/2852] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:52:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:50.290109 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:50.291811 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:50.406356 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43ce4a-76be-4347-b835-ff295728f768 service placement] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:50.412890 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:50.413598 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:50.413875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:50.414222 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:50.423491 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:50.433465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:50.435012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1456/2853] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:52:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:52:51.341811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcaee5-9a01-42ca-9141-8f7cf9dcb14c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:51.342371 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcaee5-9a01-42ca-9141-8f7cf9dcb14c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:51.342478 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcaee5-9a01-42ca-9141-8f7cf9dcb14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:51.365070 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-39bcaee5-9a01-42ca-9141-8f7cf9dcb14c None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:51.366277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-39bcaee5-9a01-42ca-9141-8f7cf9dcb14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:51.374690 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1398/2854] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 27 08:52:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:51.388715 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:51.399919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da71b5-1765-451d-9dd6-2cbfa3b8b108 service swift] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:51.404967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:51.405576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:51.405866 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:51.406122 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:51.415422 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:51.417003 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:51.426736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1457/2855] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:52:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:52:52.135332 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799031f-2f5f-405f-981d-f870c0cb801a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:52.135829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799031f-2f5f-405f-981d-f870c0cb801a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:52.136137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799031f-2f5f-405f-981d-f870c0cb801a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:52.162224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f799031f-2f5f-405f-981d-f870c0cb801a None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:52.162224 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f799031f-2f5f-405f-981d-f870c0cb801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:52.170023 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1399/2856] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 27 08:52:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:52.183481 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:52.194463 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d7a59-b8d4-4cdc-bb75-e4a635f77f1a service swift] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:52.199630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:52.200235 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:52.200517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:52.200789 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:52.210099 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:52.215611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:52.220391 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1458/2857] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:52:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:52:52.851983 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeee99-c31c-4f7e-8590-b9807f6b5652 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:52.852553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeee99-c31c-4f7e-8590-b9807f6b5652 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:52.852553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aeee99-c31c-4f7e-8590-b9807f6b5652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:52.871324 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c8aeee99-c31c-4f7e-8590-b9807f6b5652 None None] MFA Rules not processed for user `03d7fd886b4843efa9c022741bb85542`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:52.872722 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c8aeee99-c31c-4f7e-8590-b9807f6b5652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:52.880044 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1400/2858] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 27 08:52:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:52.893512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:52.905428 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b943fca-613e-47a0-bfa1-414be3308279 service swift] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:52.910530 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55bada6-d663-4020-80e0-9e153c3c2137 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92173e230c3e45c3b769221fb4b3d666', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:52.911134 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bada6-d663-4020-80e0-9e153c3c2137 service swift] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:52.911411 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bada6-d663-4020-80e0-9e153c3c2137 service swift] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:52.911677 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55bada6-d663-4020-80e0-9e153c3c2137 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:52.920427 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bada6-d663-4020-80e0-9e153c3c2137 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:52.922023 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55bada6-d663-4020-80e0-9e153c3c2137 service swift] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:52.930426 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1459/2859] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 27 08:52:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 27 08:52:55.735480 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeee99-c31c-4f7e-8590-b9807f6b5652 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:55.801318 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aeee99-c31c-4f7e-8590-b9807f6b5652 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:55.806053 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:55.806720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:55.806999 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:55.807271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] PATH_INFO: `/v3/users/c9a0f644e9794e51b843952b3552cd22` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:55.808443 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a0f644e9794e51b843952b3552cd22)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:55.810379 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:56.007410 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a0f644e9794e51b843952b3552cd22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.013656 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:56.022047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a0f644e9794e51b843952b3552cd22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.200290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a0f644e9794e51b843952b3552cd22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.208529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a0f644e9794e51b843952b3552cd22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.209251 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a0f644e9794e51b843952b3552cd22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.216903 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1401/2860] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:52:55 2022] DELETE /identity/v3/users/c9a0f644e9794e51b843952b3552cd22 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:56.221879 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:56.222548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:56.222548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:56.242869 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:56.244131 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:56.259278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:56.313104 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1460/2861] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:56.325308 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:56.327725 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:56.404383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8a0d5-b918-4955-a377-33d4f47771d3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:56.407987 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28754d92-803c-4051-9820-72880bfefe19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:56.409256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28754d92-803c-4051-9820-72880bfefe19 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:56.409256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28754d92-803c-4051-9820-72880bfefe19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:56.409256 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28754d92-803c-4051-9820-72880bfefe19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:56.420858 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28754d92-803c-4051-9820-72880bfefe19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:56.422359 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28754d92-803c-4051-9820-72880bfefe19 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:56.432902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1402/2862] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:56.706634 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:56.709948 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:56.733203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ada5-f639-4da1-8236-2dc969eada5b None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:56.733203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:56.733203 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:56.733512 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:56.733596 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] PATH_INFO: `/v3/projects/23200086cd3246318074e7eafa118e58` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:56.735022 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23200086cd3246318074e7eafa118e58)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:56.737044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:56.871050 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23200086cd3246318074e7eafa118e58', 'request_id': 'req-a1c896dd-1fcd-4ae5-b7d4-f09494454754'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.876904 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23200086cd3246318074e7eafa118e58', 'request_id': 'req-a1c896dd-1fcd-4ae5-b7d4-f09494454754'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.948883 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23200086cd3246318074e7eafa118e58', 'request_id': 'req-a1c896dd-1fcd-4ae5-b7d4-f09494454754'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.963148 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] The token cache is being invalidate because project 23200086cd3246318074e7eafa118e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:52:56.963518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.964376 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-a1c896dd-1fcd-4ae5-b7d4-f09494454754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:52:56.965887 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1461/2863] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:52:56 2022] DELETE /identity/v3/projects/23200086cd3246318074e7eafa118e58 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:56.971293 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09b6b7-d770-46de-8193-b06448640efc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:56.971837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09b6b7-d770-46de-8193-b06448640efc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:56.971837 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09b6b7-d770-46de-8193-b06448640efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:56.993286 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5d09b6b7-d770-46de-8193-b06448640efc None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:56.994818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d09b6b7-d770-46de-8193-b06448640efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:57.058009 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1403/2864] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:57.066029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17f488-eae8-4e33-951e-005dfe6054fb None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.066345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17f488-eae8-4e33-951e-005dfe6054fb None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.066689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17f488-eae8-4e33-951e-005dfe6054fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.088709 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-2d17f488-eae8-4e33-951e-005dfe6054fb None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:57.090067 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-2d17f488-eae8-4e33-951e-005dfe6054fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:57.098517 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1462/2865] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:57.103155 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09b6b7-d770-46de-8193-b06448640efc None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:57.114320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09b6b7-d770-46de-8193-b06448640efc None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:57.125192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:57.125192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.125192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.125192 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.125661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:52:57.125661 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:57.129072 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:57.143558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1404/2866] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:52:57.148555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17f488-eae8-4e33-951e-005dfe6054fb None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:57.159557 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17f488-eae8-4e33-951e-005dfe6054fb None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:57.166029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:57.166029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.166309 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.166719 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.167541 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:57.169783 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:57.294196 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1463/2867] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:52:57 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:57.298935 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:57.310846 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa350bc-f07c-4cab-814a-95c60417d13c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:57.382344 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:57.382873 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.383147 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.383416 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.384081 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:57.386001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:57.395226 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:57.612019 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1405/2868] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:52:57 2022] POST /identity/v3/users => generated 291 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:52:57.616775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:57.627858 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cffff-0364-4b9d-8854-527acda91359 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:57.632706 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:57.633256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.633532 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.633803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.634539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:57.636138 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:57.647201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1464/2869] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:57.651329 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:57.661829 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e3222-9676-4e12-9d49-dc74df1e668e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:57.667516 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e510c27-a671-4359-8951-1bd961350c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:57.668143 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510c27-a671-4359-8951-1bd961350c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.668423 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510c27-a671-4359-8951-1bd961350c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.668699 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e510c27-a671-4359-8951-1bd961350c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.669400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e510c27-a671-4359-8951-1bd961350c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:57.671041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e510c27-a671-4359-8951-1bd961350c26 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:57.684626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1406/2870] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:52:57.688946 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:57.699694 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c54f7-dcc4-484d-8323-00c731ef80e3 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:57.704585 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:57.705256 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.705537 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.705814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] PATH_INFO: `/v3/projects/d79bf8b3761f48c79c09b2cf28a00ba1/users/059172e7a08e486d9f1243b82259f10d/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.724397 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d79bf8b3761f48c79c09b2cf28a00ba1, user_id=059172e7a08e486d9f1243b82259f10d, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:57.727935 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:57.809225 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1465/2871] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:52:57 2022] PUT /identity/v3/projects/d79bf8b3761f48c79c09b2cf28a00ba1/users/059172e7a08e486d9f1243b82259f10d/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:52:57.814626 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564792-21d0-4904-a271-351a084213cf None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.814912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564792-21d0-4904-a271-351a084213cf None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.815210 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564792-21d0-4904-a271-351a084213cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.835666 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-54564792-21d0-4904-a271-351a084213cf None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:57.836913 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54564792-21d0-4904-a271-351a084213cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:57.903377 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1407/2872] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:52:57.914762 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:57.916660 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:57.964772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:52:57.986355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6921ec2-332b-43c6-8ed8-802dc60371b7 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:57.988393 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:57.989021 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:57.989351 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:57.989563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:57.999974 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:58.001432 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:58.021181 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1466/2873] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:52:59.448605 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:59.449060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:59.449060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:59.482572 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] MFA Rules not processed for user `059172e7a08e486d9f1243b82259f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:52:59.483864 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:59.559553 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1408/2874] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:52:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:52:59.571272 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:52:59.573278 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:52:59.635154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021c65-10a2-4765-9059-f5f09450a1cd service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:52:59.638814 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557ae10-b3b6-426f-b21e-f0a047e11810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:52:59.639436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ae10-b3b6-426f-b21e-f0a047e11810 service cinder] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:52:59.639693 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ae10-b3b6-426f-b21e-f0a047e11810 service cinder] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:52:59.639967 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ae10-b3b6-426f-b21e-f0a047e11810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:52:59.649302 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557ae10-b3b6-426f-b21e-f0a047e11810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:52:59.650780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557ae10-b3b6-426f-b21e-f0a047e11810 service cinder] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:52:59.659721 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1467/2875] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:52:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:53:01.682696 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:01.684182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:01.698617 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2e93-7a61-44c3-aab4-eed625b417ee None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:01.703355 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:01.703993 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:01.704268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:01.704528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] PATH_INFO: `/v3/users/edb4ee9715d946e6bd520cb5622162b7` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:01.705721 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb4ee9715d946e6bd520cb5622162b7)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:01.707825 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:01.930975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb4ee9715d946e6bd520cb5622162b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:01.946964 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb4ee9715d946e6bd520cb5622162b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:01.949160 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:53:02.051368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb4ee9715d946e6bd520cb5622162b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:02.059482 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb4ee9715d946e6bd520cb5622162b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:02.060597 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb4ee9715d946e6bd520cb5622162b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:02.067493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1409/2876] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:53:01 2022] DELETE /identity/v3/users/edb4ee9715d946e6bd520cb5622162b7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:53:02.072809 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623af2c7-4f8e-4712-8479-4bed02ae0e6a None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:02.073299 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623af2c7-4f8e-4712-8479-4bed02ae0e6a None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:02.073435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623af2c7-4f8e-4712-8479-4bed02ae0e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:02.094111 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-623af2c7-4f8e-4712-8479-4bed02ae0e6a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:53:02.095370 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-623af2c7-4f8e-4712-8479-4bed02ae0e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:02.175952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1468/2877] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:53:02.186934 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:02.188277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:02.250662 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac8b42-ab6d-4164-b3df-42a12a46afcc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:02.254457 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:02.255071 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:02.255347 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:02.255627 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:02.266442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:02.267892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:02.278885 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1410/2878] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:53:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:53:02.643210 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:02.643525 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:02.643772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:02.665745 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] MFA Rules not processed for user `059172e7a08e486d9f1243b82259f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:53:02.675018 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:02.724186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1469/2879] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:53:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:53:02.769268 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:02.771032 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:02.794860 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77690d-7bfc-4981-8506-35ff4aeba24c service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:02.801493 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:02.802320 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:02.802720 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:02.803118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] PATH_INFO: `/v3/projects/23575e72cc5443f682b5387dd62305a5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:02.805307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23575e72cc5443f682b5387dd62305a5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:02.807612 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:02.969447 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23575e72cc5443f682b5387dd62305a5', 'request_id': 'req-4bb7a510-c1bc-4983-84e4-11cdb4d19333'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:02.977503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23575e72cc5443f682b5387dd62305a5', 'request_id': 'req-4bb7a510-c1bc-4983-84e4-11cdb4d19333'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:02.980951 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:02.982418 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:02.985061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23575e72cc5443f682b5387dd62305a5', 'request_id': 'req-4bb7a510-c1bc-4983-84e4-11cdb4d19333'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:03.017291 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e7339-d722-4ef4-a9fd-6c667ce2fc25 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:03.030757 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=100 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:53:03.055874 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566bcb0-642e-46f9-8d3b-4bd798061e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:03.056586 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566bcb0-642e-46f9-8d3b-4bd798061e87 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.056895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566bcb0-642e-46f9-8d3b-4bd798061e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.057202 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566bcb0-642e-46f9-8d3b-4bd798061e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.059118 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5566bcb0-642e-46f9-8d3b-4bd798061e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:03.069895 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566bcb0-642e-46f9-8d3b-4bd798061e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:03.071407 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566bcb0-642e-46f9-8d3b-4bd798061e87 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:03.080384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1470/2880] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:53:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:53:03.081520 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] The token cache is being invalidate because project 23575e72cc5443f682b5387dd62305a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:53:03.085957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:03.085957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4bb7a510-c1bc-4983-84e4-11cdb4d19333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:53:03.089335 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1411/2881] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:53:02 2022] DELETE /identity/v3/projects/23575e72cc5443f682b5387dd62305a5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:53:03.090856 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4895608-9e5c-43c8-bc9b-480df8de460c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.091143 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4895608-9e5c-43c8-bc9b-480df8de460c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.091425 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4895608-9e5c-43c8-bc9b-480df8de460c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.113307 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-b4895608-9e5c-43c8-bc9b-480df8de460c None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:53:03.114567 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-b4895608-9e5c-43c8-bc9b-480df8de460c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:03.189476 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1471/2882] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:53:03.199014 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd306f4-fb1c-4ae6-9dba-34e3bceb5c5a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.199302 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd306f4-fb1c-4ae6-9dba-34e3bceb5c5a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.199582 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd306f4-fb1c-4ae6-9dba-34e3bceb5c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.223611 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ccd306f4-fb1c-4ae6-9dba-34e3bceb5c5a None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:53:03.225577 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ccd306f4-fb1c-4ae6-9dba-34e3bceb5c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:03.235034 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1412/2883] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 27 08:53:03.244436 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4895608-9e5c-43c8-bc9b-480df8de460c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:03.255421 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4895608-9e5c-43c8-bc9b-480df8de460c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:03.260477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:03.261100 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.261414 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.261689 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] PATH_INFO: `/v3/domains` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.264643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80290) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 27 08:53:03.264643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:03.264643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:03.281252 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1472/2884] 10.222.0.40 () {58 vars in 1153 bytes} [Wed Jul 27 08:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 27 08:53:03.282306 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd306f4-fb1c-4ae6-9dba-34e3bceb5c5a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:03.293166 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd306f4-fb1c-4ae6-9dba-34e3bceb5c5a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:03.298967 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:03.299529 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.299806 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.300183 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.309880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:03.309880 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:03.398703 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1413/2885] 10.222.0.40 () {60 vars in 1153 bytes} [Wed Jul 27 08:53:03 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:53:03.403632 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:03.416455 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240eed8-c507-4fcc-96b5-001de967eccb admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:03.493822 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:03.494919 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.495433 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.495987 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] PATH_INFO: `/v3/users` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.497047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:03.499966 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:03.654077 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1473/2886] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 27 08:53:03 2022] POST /identity/v3/users => generated 282 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 27 08:53:03.658120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:03.670001 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81deff-3c0e-4ac6-b018-fe3667aa1da6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:03.675029 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:03.679363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.679363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.679363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.679363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:03.679363 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:03.690749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1414/2887] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:53:03.695875 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:03.710827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55fccd-c4cd-4548-a1ad-21cc3cea2962 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:03.715803 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351784b-3d62-48ed-9fbd-d9bc62595063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:03.716763 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351784b-3d62-48ed-9fbd-d9bc62595063 admin admin] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.717257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351784b-3d62-48ed-9fbd-d9bc62595063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.717548 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351784b-3d62-48ed-9fbd-d9bc62595063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.718538 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351784b-3d62-48ed-9fbd-d9bc62595063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:03.720156 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351784b-3d62-48ed-9fbd-d9bc62595063 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:03.731563 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1474/2888] 10.222.0.40 () {58 vars in 1122 bytes} [Wed Jul 27 08:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 27 08:53:03.738538 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:03.749307 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72accadc-1789-4f15-9985-6a983c8974a2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:03.753944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:03.754639 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.754917 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.755189 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] PATH_INFO: `/v3/projects/05023c7f714e41ed8d38017215abfa5d/users/7339d5a0ab364560b8b27c93f258824a/roles/6a60961045ef46cd91a802e9a6cf7b9e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.774689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05023c7f714e41ed8d38017215abfa5d, user_id=7339d5a0ab364560b8b27c93f258824a, role_id=6a60961045ef46cd91a802e9a6cf7b9e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:03.778422 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:03.875253 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1415/2889] 10.222.0.40 () {60 vars in 1483 bytes} [Wed Jul 27 08:53:03 2022] PUT /identity/v3/projects/05023c7f714e41ed8d38017215abfa5d/users/7339d5a0ab364560b8b27c93f258824a/roles/6a60961045ef46cd91a802e9a6cf7b9e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 27 08:53:03.887527 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ecb7-41ff-4f34-b512-9fea15cb809c None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:03.887796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ecb7-41ff-4f34-b512-9fea15cb809c None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:03.888110 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4ecb7-41ff-4f34-b512-9fea15cb809c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:03.923970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-83a4ecb7-41ff-4f34-b512-9fea15cb809c None None] MFA Rules not processed for user `7339d5a0ab364560b8b27c93f258824a`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:53:03.925748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-83a4ecb7-41ff-4f34-b512-9fea15cb809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:03.970095 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:53:04.012207 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1475/2890] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:53:03 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:53:04.024531 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:53:04.026122 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:53:04.033264 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:53:04.092918 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b9fdf-ef2f-48cd-b99e-08c605766c88 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:53:04.097059 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5467da1-1bdf-4d38-a7fb-3a17670925b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:53:04.097689 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5467da1-1bdf-4d38-a7fb-3a17670925b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:53:04.097975 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5467da1-1bdf-4d38-a7fb-3a17670925b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:53:04.098303 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5467da1-1bdf-4d38-a7fb-3a17670925b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:53:04.108459 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5467da1-1bdf-4d38-a7fb-3a17670925b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:53:04.109893 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5467da1-1bdf-4d38-a7fb-3a17670925b7 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:53:04.120972 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1416/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:53:04 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:53:14.057382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=14 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:53:14.194383 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=24 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:54:00.940748 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4ecb7-41ff-4f34-b512-9fea15cb809c None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:54:01.003970 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4ecb7-41ff-4f34-b512-9fea15cb809c None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:54:01.008643 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:54:01.009280 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:01.009555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:01.009827 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] PATH_INFO: `/v3/users/7339d5a0ab364560b8b27c93f258824a` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:01.011029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7339d5a0ab364560b8b27c93f258824a)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:54:01.012952 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:54:01.189154 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7339d5a0ab364560b8b27c93f258824a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:01.201631 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7339d5a0ab364560b8b27c93f258824a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:01.271698 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7339d5a0ab364560b8b27c93f258824a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:01.272310 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7339d5a0ab364560b8b27c93f258824a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:01.272918 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7339d5a0ab364560b8b27c93f258824a'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:01.286435 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1476/2892] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:54:00 2022] DELETE /identity/v3/users/7339d5a0ab364560b8b27c93f258824a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:54:01.292755 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:01.293317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:01.293317 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:01.312357 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] MFA Rules not processed for user `c247fd0f0c364a67b9e7168fc38f6873`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:54:01.313565 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:01.372544 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1417/2893] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:54:01.382296 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:54:01.384585 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:01.443315 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcb2ce-1322-4f9d-ac95-26b3507b5704 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:54:01.447417 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:54:01.448063 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:01.448350 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:01.448612 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:01.457555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:54:01.459015 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:54:01.470244 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1477/2894] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:54:01.891643 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:54:01.894528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:01.906242 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da0a25-085b-4ec4-8779-04156264de52 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:54:01.911060 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:54:01.911634 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:01.911926 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:01.912182 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] PATH_INFO: `/v3/projects/05023c7f714e41ed8d38017215abfa5d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:01.913340 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05023c7f714e41ed8d38017215abfa5d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:54:01.915842 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:54:02.015834 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05023c7f714e41ed8d38017215abfa5d', 'request_id': 'req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:02.021304 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05023c7f714e41ed8d38017215abfa5d', 'request_id': 'req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:02.028067 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05023c7f714e41ed8d38017215abfa5d', 'request_id': 'req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:02.094800 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] The token cache is being invalidate because project 05023c7f714e41ed8d38017215abfa5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:54:02.095084 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:02.095760 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:54:02.097296 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1418/2895] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:54:01 2022] DELETE /identity/v3/projects/05023c7f714e41ed8d38017215abfa5d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:54:11.465714 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:54:12.091282 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:54:30.913188 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:54:30.924796 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1018-7ef4-43d3-9b67-c21658c778a3 service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:54:30.985345 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e76e5-b191-492d-b738-dbfd8b8f7ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:54:30.989141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e76e5-b191-492d-b738-dbfd8b8f7ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:30.989141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e76e5-b191-492d-b738-dbfd8b8f7ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:30.989141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e76e5-b191-492d-b738-dbfd8b8f7ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:30.989141 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dc0e76e5-b191-492d-b738-dbfd8b8f7ab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:31.064400 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e76e5-b191-492d-b738-dbfd8b8f7ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:54:31.065923 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e76e5-b191-492d-b738-dbfd8b8f7ab3 service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:54:31.078775 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1478/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:54:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:54:35.970442 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:54:35.972154 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:35.986346 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ceca-4dde-448f-9f5a-e6f7fd4f7203 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:54:35.990128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237144f2-736b-4042-be3e-9699c9963e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:54:35.990774 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237144f2-736b-4042-be3e-9699c9963e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:35.991051 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237144f2-736b-4042-be3e-9699c9963e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:35.991316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237144f2-736b-4042-be3e-9699c9963e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:35.993146 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-237144f2-736b-4042-be3e-9699c9963e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:36.008388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237144f2-736b-4042-be3e-9699c9963e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:54:36.009833 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237144f2-736b-4042-be3e-9699c9963e49 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:54:36.021523 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1419/2897] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:54:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:54:38.369268 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:38.369817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:38.369817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:38.390605 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] MFA Rules not processed for user `059172e7a08e486d9f1243b82259f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:54:38.391817 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:38.451013 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1479/2898] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:54:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:54:39.365407 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:54:39.365931 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:54:39.366030 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:54:39.386646 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] MFA Rules not processed for user `3023e29a392a4d97acbb11faecc0c9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:54:39.399811 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:54:39.458429 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1420/2899] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:54:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:54:48.444123 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=79 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:54:49.440058 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=31 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:55:02.881940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:55:02.884017 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:55:02.905726 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13d3b2-b8fa-4004-844c-75470f8bcfce None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:55:02.912972 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:55:02.913642 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:55:02.913922 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:55:02.914226 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:55:02.916086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:55:02.930184 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:55:02.931780 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:55:02.945029 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1480/2900] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:55:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:55:12.939630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=6 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:55:56.041413 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:55:56.042868 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:55:56.176874 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b351d-ad88-405c-a6b8-c8a4c9e645f2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:55:56.182559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:55:56.183190 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:55:56.183503 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:55:56.183915 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:55:56.187368 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:55:56.200912 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:55:56.202549 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:55:56.215977 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1421/2901] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:55:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:55:56.216845 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:55:56.218994 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:55:56.356175 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb5fb-46f6-4879-a0cb-649abf1f5118 service nova] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:55:56.360704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:55:56.361382 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:55:56.361753 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:55:56.362049 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] PATH_INFO: `/v3/users/059172e7a08e486d9f1243b82259f10d` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:55:56.363838 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=059172e7a08e486d9f1243b82259f10d)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:55:56.365761 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:55:56.620576 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059172e7a08e486d9f1243b82259f10d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:56.633007 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059172e7a08e486d9f1243b82259f10d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:56.703824 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059172e7a08e486d9f1243b82259f10d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:56.704448 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059172e7a08e486d9f1243b82259f10d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:56.705057 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059172e7a08e486d9f1243b82259f10d'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:56.719044 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1481/2902] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:55:56 2022] DELETE /identity/v3/users/059172e7a08e486d9f1243b82259f10d => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:55:56.985451 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:55:56.998446 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70370c91-f115-4f1e-9f74-439a1111dc4f service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:55:57.057776 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:55:57.058542 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:55:57.058654 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:55:57.058910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] PATH_INFO: `/v3/projects/d79bf8b3761f48c79c09b2cf28a00ba1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:55:57.060061 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d79bf8b3761f48c79c09b2cf28a00ba1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:55:57.061944 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:55:57.183047 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:55:57.187078 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd79bf8b3761f48c79c09b2cf28a00ba1', 'request_id': 'req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:57.193817 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd79bf8b3761f48c79c09b2cf28a00ba1', 'request_id': 'req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:57.200910 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd79bf8b3761f48c79c09b2cf28a00ba1', 'request_id': 'req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:57.297998 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] The token cache is being invalidate because project d79bf8b3761f48c79c09b2cf28a00ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:55:57.298316 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:57.298958 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:55:57.300499 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1422/2903] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:55:56 2022] DELETE /identity/v3/projects/d79bf8b3761f48c79c09b2cf28a00ba1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:56:06.715249 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:56:07.304455 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:56:35.460485 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:56:35.462311 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:56:35.547531 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef994a3-d130-4ad6-bf29-cafb9c1065dd admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:56:35.601384 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a6e9e-002b-4db8-8638-d80a37baa0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:56:35.601968 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a6e9e-002b-4db8-8638-d80a37baa0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:56:35.602279 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a6e9e-002b-4db8-8638-d80a37baa0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:56:35.602555 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a6e9e-002b-4db8-8638-d80a37baa0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:56:35.604518 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-4b6a6e9e-002b-4db8-8638-d80a37baa0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:56:35.614277 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a6e9e-002b-4db8-8638-d80a37baa0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:56:35.615701 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a6e9e-002b-4db8-8638-d80a37baa0ba service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:56:35.623769 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1482/2904] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:56:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:56:45.620101 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=55 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:56:52.418810 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:56:52.420218 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:56:52.503567 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fb974-6bbf-41bb-93ae-3e52c6d8ad3e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:56:52.507176 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a29a23f8a84076bab3adefc042a088', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:56:52.507797 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:56:52.508073 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:56:52.508343 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:56:52.510102 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:56:52.573970 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:56:52.576957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:56:52.587870 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1423/2905] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:56:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:57:02.582984 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=57 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:57:04.307711 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] REQUEST_METHOD: `POST` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:57:04.308324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:57:04.308324 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:57:04.327377 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.auth.core [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] MFA Rules not processed for user `3023e29a392a4d97acbb11faecc0c9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80290) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 27 08:57:04.328736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:04.336240 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1483/2906] 10.222.0.40 () {58 vars in 958 bytes} [Wed Jul 27 08:57:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 27 08:57:04.716729 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:57:04.719300 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:04.731559 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0f04e-2ebb-403c-8e69-2d97f654787e service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:57:04.735275 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:57:04.735836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:57:04.736118 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:57:04.736388 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:57:04.738232 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:04.747906 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:57:04.749889 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:57:04.759045 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1424/2907] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:57:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:57:05.526842 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:57:05.528604 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:05.616707 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80367bb9-f386-43d0-a94f-7510e74e5de0 None None] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:57:05.621545 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:57:05.622182 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:57:05.622465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:57:05.622738 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:57:05.624772 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:05.637092 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:57:05.638739 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:57:05.649519 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1484/2908] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:57:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:57:14.755041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:57:15.644949 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=35 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:57:20.968748 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:57:20.970749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:21.051574 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3ce57-5aee-4fd9-976d-e86eb8c8732f service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:57:21.058092 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:57:21.058957 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:57:21.059233 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:57:21.059501 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:57:21.061345 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:21.075356 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:57:21.077827 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:57:21.089120 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1425/2909] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:57:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:57:24.711112 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:57:24.712649 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:24.725465 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff3a1-952a-4db4-a7b9-b36950e2c978 service placement] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:57:24.729257 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:57:24.729891 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:57:24.730201 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:57:24.730482 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:57:24.732509 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:24.800959 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:57:24.802472 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:57:24.813142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1485/2910] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:57:27.034333 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:57:27.035830 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:27.049884 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9260e-d51a-4abf-8278-e4410e7d05f9 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:57:27.058528 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e4e461b0b6142b38fd7de56395733b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:57:27.061904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:57:27.061904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:57:27.061904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:57:27.061904 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:57:27.076130 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:57:27.079558 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:57:27.092749 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1426/2911] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 27 08:57:34.808227 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=30 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:57:37.088570 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=40 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:58:33.789609 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:58:33.791004 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:58:33.828543 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57a2e5-70e8-4f57-8756-2c0608b5c61d service neutron] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:58:33.833736 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:58:33.834394 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:58:33.834685 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:58:33.835047 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:58:33.836754 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:58:33.849801 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:58:33.851434 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:58:33.861992 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1486/2912] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:58:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:58:43.857216 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=15 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:59:30.795396 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:59:30.805921 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acc208-bc94-489b-a10d-1f0115621921 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:59:30.809669 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c308c33c674f1ea7111c0297174c2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:59:30.810389 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:59:30.810839 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:59:30.811271 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:59:30.813290 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:59:30.888325 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:59:30.889782 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:59:30.899876 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1427/2913] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:59:34.495601 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:59:34.506380 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd75a4-3728-4269-ad0c-b5189874842f service placement] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:59:34.512833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:59:34.512833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] REQUEST_METHOD: `GET` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:59:34.512833 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:59:34.513179 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:59:34.514812 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:59:34.527539 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:59:34.529152 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:59:34.540373 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1487/2914] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 08:59:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 08:59:38.397364 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:59:38.399216 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:59:38.411836 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd6c20-058e-4983-b4ea-9ade044f60d6 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:59:38.418604 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:59:38.419277 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:59:38.419560 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:59:38.419892 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] PATH_INFO: `/v3/users/3023e29a392a4d97acbb11faecc0c9b8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:59:38.421135 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=3023e29a392a4d97acbb11faecc0c9b8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:59:38.423162 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:59:38.581939 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3023e29a392a4d97acbb11faecc0c9b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:38.594698 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3023e29a392a4d97acbb11faecc0c9b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:38.636161 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3023e29a392a4d97acbb11faecc0c9b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:38.643704 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3023e29a392a4d97acbb11faecc0c9b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:38.644400 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3023e29a392a4d97acbb11faecc0c9b8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:38.650649 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1428/2915] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:59:38 2022] DELETE /identity/v3/users/3023e29a392a4d97acbb11faecc0c9b8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:59:38.911012 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:59:38.922823 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef3a0b-3475-44f9-b4aa-73bf17803f91 service placement] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:59:38.980437 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:59:38.981086 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:59:38.981355 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:59:38.981630 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] PATH_INFO: `/v3/projects/e664288b45ae4b98ae22270ea7eac1e6` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:59:38.985094 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e664288b45ae4b98ae22270ea7eac1e6)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:59:38.986947 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:59:39.088611 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e664288b45ae4b98ae22270ea7eac1e6', 'request_id': 'req-7e09fe4b-aa13-440a-9202-7ffb2b21d977'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.094624 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e664288b45ae4b98ae22270ea7eac1e6', 'request_id': 'req-7e09fe4b-aa13-440a-9202-7ffb2b21d977'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.200788 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e664288b45ae4b98ae22270ea7eac1e6', 'request_id': 'req-7e09fe4b-aa13-440a-9202-7ffb2b21d977'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.214167 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] The token cache is being invalidate because project e664288b45ae4b98ae22270ea7eac1e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:59:39.214451 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.215142 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.216704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1488/2916] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:59:38 2022] DELETE /identity/v3/projects/e664288b45ae4b98ae22270ea7eac1e6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:59:39.221399 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:59:39.223114 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:59:39.237930 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d0141-529d-44e1-84fd-8fbaaf7952ac admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:59:39.300496 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:59:39.301128 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:59:39.301404 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:59:39.301673 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] PATH_INFO: `/v3/users/984353383e614d638fbb0eb15cd944a2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:59:39.302900 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=984353383e614d638fbb0eb15cd944a2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:59:39.304976 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:59:39.312199 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:59:39.523453 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '984353383e614d638fbb0eb15cd944a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.535768 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '984353383e614d638fbb0eb15cd944a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.623677 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '984353383e614d638fbb0eb15cd944a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.631228 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '984353383e614d638fbb0eb15cd944a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.631887 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '984353383e614d638fbb0eb15cd944a2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:39.638066 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1429/2917] 10.222.0.40 () {58 vars in 1224 bytes} [Wed Jul 27 08:59:39 2022] DELETE /identity/v3/users/984353383e614d638fbb0eb15cd944a2 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:59:39.881750 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Authenticating user token {{(pid=80290) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 08:59:39.883603 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.fernet_utils [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80290) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 08:59:39.897662 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09fe4b-aa13-440a-9202-7ffb2b21d977 admin admin] Validating token access rules against request {{(pid=80290) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 08:59:39.955993 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c247fd0f0c364a67b9e7168fc38f6873', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28a37ee923b244c5bc101a0c6c5cd5d6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80290) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 08:59:39.956704 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 08:59:39.956910 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 08:59:39.957186 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] PATH_INFO: `/v3/projects/f02877d9307944d78ebe93ea3d5b9dfb` {{(pid=80290) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 08:59:39.958477 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02877d9307944d78ebe93ea3d5b9dfb)` {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 08:59:39.961035 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] RBAC: Authorization granted {{(pid=80290) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 08:59:39.983955 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:59:40.065192 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02877d9307944d78ebe93ea3d5b9dfb', 'request_id': 'req-1436b634-436c-4a4c-a018-9c8ebea420cf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:40.070595 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02877d9307944d78ebe93ea3d5b9dfb', 'request_id': 'req-1436b634-436c-4a4c-a018-9c8ebea420cf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:40.126558 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02877d9307944d78ebe93ea3d5b9dfb', 'request_id': 'req-1436b634-436c-4a4c-a018-9c8ebea420cf'} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:40.139462 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] The token cache is being invalidate because project f02877d9307944d78ebe93ea3d5b9dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80290) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 27 08:59:40.139751 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:40.140410 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG keystone.notifications [None req-1436b634-436c-4a4c-a018-9c8ebea420cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80290) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 27 08:59:40.141940 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: [pid: 80290|app: 0|req: 1489/2918] 10.222.0.40 () {58 vars in 1233 bytes} [Wed Jul 27 08:59:39 2022] DELETE /identity/v3/projects/f02877d9307944d78ebe93ea3d5b9dfb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 27 08:59:49.731852 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 08:59:50.136424 dv-cinder-851170-2-iscsi devstack@keystone.service[80290]: DEBUG dbcounter [-] [80290] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80290) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 27 09:03:37.743460 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 27 09:03:37.744850 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 09:03:37.816137 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82164144-20cc-47fe-9f9f-c1e035c65bcf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 27 09:03:37.875818 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788960f-679f-41bb-8497-529a5a0a9fd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04d05e0dfd474dfdb848b129941532b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e9d120f709a1446e9614a10cfc7b01e2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 27 09:03:37.876438 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788960f-679f-41bb-8497-529a5a0a9fd7 service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 27 09:03:37.876719 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788960f-679f-41bb-8497-529a5a0a9fd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 27 09:03:37.876994 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788960f-679f-41bb-8497-529a5a0a9fd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 27 09:03:37.878787 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c788960f-679f-41bb-8497-529a5a0a9fd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 27 09:03:37.891221 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788960f-679f-41bb-8497-529a5a0a9fd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 27 09:03:37.893041 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788960f-679f-41bb-8497-529a5a0a9fd7 service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 27 09:03:37.903625 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1430/2919] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 27 09:03:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 27 09:03:47.898902 dv-cinder-851170-2-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=58 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}